We are often told by clients that they want a security audit. The word “audit” is often inappropriate. Here are five types of security evaluations and their goals and audiences:
1) Vulnerability Tests
2) Penetration Tests
3) Risk Assessments
4) Compliance Audits
5) Due Diligence Questionnaires
Vulnerability Tests
A vulnerability is a “flaw or weakness in system security procedures, design, implementation, or internal controls that could be exercised (accidentally triggered or intentionally exploited) and result in a security breach or a violation of the system’s security policy.” Vulnerability tests check for the existence of known flaws. Here are some useful characteristics of vulnerabilities:
1) There is a fix
2) There is a no fix
3) There is an exploit available
4) There is no known exploit available
Vulnerabilities for which there is no fix and there are known exploits are causes for immediate mitigation. The vulnerability tester may provide a list of mitigations as part of their analysis. A vulnerability test ought to be done before a system goes live to ensure that no flaw exposes the systems despite security controls. Vulnerability tests can be performed externally—inside or outside the organization’s firewalls—or on the system itself. The results will be significantly different. A vulnerability test installed on and performed against the system on which it is installed will produce results that are much more extensive then the same test performed from the network or outside of the firewalls. Even the credentials used to perform the vulnerability test will affect the results. Patch updates are sometimes followed by vulnerability testing to ensure that security patches accomplished their task. Choosing where to test from and what credentials to use during the test is an important part of what the tests are intended to show. For example, if you want to show just how far an insider can get when attacking, use an insider’s credentials and test from inside the network or from approved remote access. On the other hand, if you want to show what a random Internet attacker can see, test from the Internet and without any special credentials.
GOAL: The goal of vulnerability tests is to show what still needs to be corrected to meet the organizations security controls.
AUDIENCE: Executive management may only want to see pass/fail results. Technical staff wants the details so they can correct. IT Management needs the results both to show they are doing their jobs and to check on their own staff and vendors performing the work.
LEGAL IMPLICATIONS OF TEST:
Once a vulnerability is known, the organization is on notice about it. This means that if there is some resulting damage because of failure to mitigate, then the organization can be liable for the damage. Don’t refuse to have the test for this reason though. There is similar liability for “should have known”.
Testers might be violating their terms of service of their own Internet connecting by performing the tests. Be sure that that the agreement with the tester places liability on them for any actions by their own Internet providers for performing the test.
Be sure that testers are clear on which systems and networks to test. Some tests can lock or reboot a system and if this is done to someone else’s systems at your request then your organization becomes liable. There is some protection if the tester is an independent contractor.
Penetration Tests
Penetrations tests are attempts to breach existing security. The scope of a penetration test can vary. Sometimes the test is for a single application on a single server. Or the entire perimeter defenses are tested to see if they are protecting assets adequately. In addition to purely electronic penetration tests, social engineering may be employed. Social engineering is a method of deceiving someone. The goal is to convince someone to perform a task that helps in attacking the organization. This may be as simple as opening an email with malware attached. It may be by getting an inside person to make changes for the attacker. In a large organization, social engineering attacks are almost always successful. People are just too easily fooled by those experienced in doing social engineering attacks. In contrast to vulnerability tests, a penetration test is also a test of the skill of the tester. It is a lot more difficult to accomplished the actual breach than to simple check if there are vulnerabilities that would permit a breach.
GOAL: The goal of a penetration test should be to show whether a reasonable attack against your organization will succeed or not.
AUDIENCE: Executive management may only want to see pass/fail results. Technical staff wants the details so they can correct. IT Management needs the results both to show they are doing their jobs and to check on their own staff and vendors performing the work.
Risk Assessments
Risk assessments are used to identify, estimate, and prioritize risk to organizational operations (i.e., mission, functions, image, and reputation), organizational assets, individuals, other organizations, and the Nation, resulting from the operation and use of information systems. The purpose of risk assessments is to inform decision makers and support risk responses by identifying: (i) relevant threats to organizations or threats directed through organizations against other organizations; (ii) vulnerabilities both internal and external to organizations;(iii) impact (i.e., harm) to organizations that may occur given the potential for threats exploiting vulnerabilities; and (iv) likelihood that harm will occur. The end result is a determination of risk (i.e., typically a function of the degree of harm and likelihood of harm occurring). Risk assessments can be conducted at all three tiers in the risk management hierarchy—including Tier 1 (organization level), Tier 2 (mission/business process level), and Tier 3 (information system level). At Tiers 1 and 2, organizations use risk assessments to evaluate, for example, systemic information security-related risks associated with organizational governance and management activities, mission/business processes, enterprise architecture, or the funding of information security programs. At Tier 3, organizations use risk assessments to more effectively support the implementation of the Risk Management Framework (i.e., security categorization;
security control selection, implementation, and assessment; information system and common control authorization; and security control monitoring).
GOAL: The goal of a risk assessment is to let management know what risks exist so that they can decide if they are willing to accept those risks or if they should invest time and money to reduce the risks.
AUDIENCE: Executive management and the board are the only ones who can really accept risk for the entire organization.
Compliance Audits
Every organization has some set of rules that must be followed. Compliance audits are intended to see if those rules are followed. From a security standpoint, which rules to test against make an enormous difference for the compliance audit. It would be almost useless to simply pick a standard and check for everything unless your organization has been through the process of setting security controls in place prior to the audit.
GOAL: The goal of a compliance audit is to show a pass/fail result of how well the organization meets the required rules tested against.
AUDIENCE: Executive management and board of directors needs assurance that their legal obligations are being met. IT Management needs to know if their controls are holding.
Due Diligence Questionnaires
Due Diligence questionnaires are a subset of compliance audits. Often, an organization does business with others that require them to show, for their own compliance, whether their partners meet or exceed the security controls they have. Every organization should be doing this with its vendors. If you handle the confidential information of another business then you may have seen due diligence questionnaires. Before creating questions for vendors, the standard of security must be chosen.
GOAL: When answering due diligence questionnaires, the goal is to answer truthfully but also not to divulge confidential information about security. Fudging on the answers puts the organization at serious risk. It’s better to answer truthfully or not at all then to mislead. When creating questionnaires for other organizations, the goal should be to ask the kinds of questions that show whether they can maintain the confidentiality, integrity and availability required by existing security policies. To do that, you need to have already set your security goals and be on the way to meeting them yourselves.
AUDIENCE: Compliance auditors, Executive and IT management.
Joel Colvin has been a security consultant since 1992 and an attorney since 2015. If you would like to know more or have a version of this presentation at your organization, please contact him at jcolvin@jcolvinlaw.com.