More Security Metrics

Although I wrote about Security Metrics: Replacing Fear, Uncertainty and Doubt by Andrew Jaquith earlier, a single post doesn't do this important topic justice. The key theme as expressed by Jaquith is

...information security is one of the few management disciplines that has yet to submit itself to serious analytic scrutiny.

This lack of analytic scrutiny in the form of security metrics makes risk management especially difficult for executive understanding and guidance, especially when discussing the necessary level of investment required. Executives ideally want their security and compliance metrics to answer the following questions:

• How effective are my security processes?
• Am I better off than I was this time last year?
• How do I compare with my peers?
• Am I spending the right amount of money?
• What are my risk transfer options?

As previously discussed, most functions within an enterprise -- HR, finance, manufacturing, supply chain, call center, e-commerce and operations -- have the ability to measure their performance by tracking key metrics, and comparing with other companies in a peer group. Such metrics share the characteristics of being simple to explain, readily lending themselves to benchmarking, and being consistently and automatically collected.

Without such metrics, we're doomed to reactive rather than proactive risk management. Or, as Jaquith calls it, we're on the hamster wheel of pain:

Here are Jaquith's suggested questions for management when measuring audit and compliance processes and their related investments:

1. How much time and effort are security staff spending on audit-related activities? (Metrics: # regulatory audits completed, time/cost of audit activities)
2. Have audits uncovered serious weaknesses in existing controls? (Metrics: % security compliance reviews with material weaknesses, % key external requirements compliant per external audit)
3. How much time and effort are security staff spending fixing problems uncovered by audits? (Metrics: # pending deficiencies and estimated time/cost to complete, time/cost spent on remediation activities)
4. Have audit activities uncovered problems with controls that would affect customer trust or privacy? (Metric: # pending customer-related deficiencies and estimated time/cost to complete)

Only by employing security metrics and submitting to serious analytic scrutiny can an enterprise get security and compliance risk management off of the hamster wheel of pain and onto a level playing field with other disciplines.

Security Metrics

Andrew Jaquith, in his book Security Metrics: Replacing Fear, Uncertainty and Doubt, describes the value of metrics in general and in doing so identifies one of the key challenges in ensuring system security:

Today's information security battleground is all about entitlements -- who's got them, whether they were granted properly, and how to enforce them.

The book describes how metrics can be applied in managing security systems in general, and in entitlements/access rights in particular. Jaquith, a senior analyst at Forrester, cites examples of how other disciplines and industries use key metrics to compare their operations to peer companies. For example, freight companies know their freight cost per mile and loading factors-as well as those of their competitors. Management can therefore set meaningful objectives and measure themselves against comparable companies. Choosing to be above, on, or below an industry average is a question of strategy as well as operational efficiency. For example, a freight company may be willing to have a lower load factor than its peers if that's the tradeoff required to offer faster delivery times (for which it presumably charges a premium).

Similarly, warehousing firms measure and compare their cost/square foot and inventory turns, and e-commerce companies measure their website conversion rates. And of course financial metrics have been standardized and reported on for years. Companies can therefore compare relevant metrics to those of their peers in order to better evaluate their internal performance.

Could such a use of metrics apply to security? And can metrics be of use in the "entitlements battleground"?

First, let's look at Jacquith's definition of a good metric:

1. consistently measured, without subjective criteria;
2. cheap to gather, preferably in an automated way;
3. expressed as a cardinal number or percentage, not with qualitative labels such as high, medium and low;
4. expressed using at least one unit of measure, such as "defects" or "dormant accounts"; and
5. contextually specific -- relevant enough to decision-makers so that they can take action.

So what about the "information security battleground", namely entitlements and access rights? What metrics are relevant to that? Jaquith lists pertinent questions and the metrics that can guide management actions, for example: Does the organization review employee entitlements? An example metric would be % accounts dormant. (The complete discussion starts on page 117 of Jaquith's book under the heading Ensuring System Security.)

One of the advantages of a multi-tenant SaaS solution is the global statistical perspective that can be provided, which allows customers to compare their performance to that of their peers. By knowing industry averages for key metrics  customers can benchmark their internal performance and security objectives to those of comparable organizations. What better way to arm oneself for the information security battleground known as entitlements management?

The definition and application of security metrics is ongoing. One resource I recommend is Securitymetrics.org, which provides empirical strategies for decision-makers and security practitioners and which includes links to digests, presentations, and handouts from past Metricon Workshops.

Is Perfect Access Control Possible?

Bruce Schneier, the Chief Security Technology Officer of BT and a highly regarded security guru, engaged in a point/counter-point debate with Marcus Ranum in an article entitled Schneier-Ranum Face-Off: Is Perfect Access Control Possible?

The question regarding the efficacy of access controls is particularly relevant today, especially in light of the fact that excessive access rights was the top audit finding over the past two years. How can that be resolved? The general consensus among Identity Management (IdM) experts is that organizations should implement a role-based access control (RBAC) system to manage access rights. But as Schneier points out:

RBAC is very hard to implement correctly. Organizations generally don't even know who has what role. The employee doesn't know, the boss doesn't know--and these days the employee might have more than one boss -- and senior management certainly doesn't know.

Ranum notes that part of the problem is that we're paying for decisions made over the past decade to make critical data easier and cheaper to access.

What both Schneier and Ranum agree on is that over-entitlement is the norm today, and these excessive access rights -- also called excessive entitlements -- represent a security and compliance exposure.

So where does that leave us? Based on what I've seen, I have to agree with Schneier's assessment:

In the end, a perfect access control system just isn't possible; organizations are simply too chaotic for it to work.

If RBAC systems are so hard to implement correctly, and even if doing so still leaves the organization with excessive access rights and their associated risks and vulnerabilities, what can be done? User activity monitoring in the form of an Identity and Access Assessment (IdAA) solution can complement RBAC identity management systems by providing feedback that uncovers excess entitlement in the form of dormant (aka zombie) accounts. Therefore, even if RBAC is very hard to implement correctly, at least the organization can gain visibility into and remove the vulnerabilities and compliance exposure associated with excessive access rights.

Cloud Security and Privacy

I wanted to discuss a newly-published book, Cloud Security and Privacy: An Enterprise Perspective on Risks and Compliance. But the book has too much valuable content to do it justice in a 500-word blog post, so I will focus today on a single chapter: Data Security and Storage.

First, props to the authors (Tim Mather, Subra Kumaraswarmy, and Shahed Latif) who have written a thoughtful, in-depth book on a topic that's often subject to hype and relatively unsatisfying sound bites. The authors aren't working an agenda, and they aren't promoting cloud services. Nor do they provide easy answers. But they do offer insights as to what the critical issues are, what questions to ask your cloud service provider (CSP) to truly assess relevant risk factors, and what strategies might be considered when your security and privacy requirements exceed the service levels provided by current cloud services.

In my discussions with customers, the biggest concerns I hear relative to public cloud services are on the subject of data security, especially data privacy and data remanence.The authors discuss aspects of data security related to data in transit and data at rest, including multitenancy issues. Here's a partial checklist: You should know whether your CSP uses vetted encryption algorithms, and whether the protocols employed ensure data integrity as well as data confidentiality. You should be aware that even when data at rest is encrypted, it can't be operated on by the application without being unencrypted -- in such a case you'll want to know whether memory, caches and temporary storage are wiped afterward (the answer is almost certainly "no", or, more likely, such questions won't be answered by the CSP). The same set of questions (and answers) apply to the issue of data migration and to processes by which failed or obsolete storage devices are decommissioned. 

The key point in this chapter is with regard to data security mitigation. How can you compensate when CSP data security capabilities are inadequate to your needs? The authors' answer: Don't put sensitive data in a public cloud, other than for simple cloud storage services where your data is (and always remains) encrypted. I couldn't agree more, although I would add that this is an area that CSPs are aware of and working on, and I predict that in the near future (2-3 years) public cloud data security will have improved substantially.

A prerequisite to evaluating whether public CSPs' security is adequate to your needs is to classify your data. Only by doing so can an organization make informed judgments as to whether the cloud security is "good enough". The organization's policy should be to limit cloud-based applications to only those that operate on low- or moderate-risk data, such as CRM and internal log data. Higher-risk data sets may be stored in the public cloud only if they have been "sanitized" (i.e. sensitive data removed or anonymized).