Security Metrics

Andrew Jaquith, in his book Security Metrics: Replacing Fear, Uncertainty and Doubt, describes the value of metrics in general and in doing so identifies one of the key challenges in ensuring system security:

Today's information security battleground is all about entitlements -- who's got them, whether they were granted properly, and how to enforce them.

The book describes how metrics can be applied in managing security systems in general, and in entitlements/access rights in particular. Jaquith, a senior analyst at Forrester, cites examples of how other disciplines and industries use key metrics to compare their operations to peer companies. For example, freight companies know their freight cost per mile and loading factors-as well as those of their competitors. Management can therefore set meaningful objectives and measure themselves against comparable companies. Choosing to be above, on, or below an industry average is a question of strategy as well as operational efficiency. For example, a freight company may be willing to have a lower load factor than its peers if that's the tradeoff required to offer faster delivery times (for which it presumably charges a premium).

Similarly, warehousing firms measure and compare their cost/square foot and inventory turns, and e-commerce companies measure their website conversion rates. And of course financial metrics have been standardized and reported on for years. Companies can therefore compare relevant metrics to those of their peers in order to better evaluate their internal performance.

Could such a use of metrics apply to security? And can metrics be of use in the "entitlements battleground"?

First, let's look at Jacquith's definition of a good metric:

1. consistently measured, without subjective criteria;
2. cheap to gather, preferably in an automated way;
3. expressed as a cardinal number or percentage, not with qualitative labels such as high, medium and low;
4. expressed using at least one unit of measure, such as "defects" or "dormant accounts"; and
5. contextually specific -- relevant enough to decision-makers so that they can take action.

So what about the "information security battleground", namely entitlements and access rights? What metrics are relevant to that? Jaquith lists pertinent questions and the metrics that can guide management actions, for example: Does the organization review employee entitlements? An example metric would be % accounts dormant. (The complete discussion starts on page 117 of Jaquith's book under the heading Ensuring System Security.)

One of the advantages of a multi-tenant SaaS solution is the global statistical perspective that can be provided, which allows customers to compare their performance to that of their peers. By knowing industry averages for key metrics  customers can benchmark their internal performance and security objectives to those of comparable organizations. What better way to arm oneself for the information security battleground known as entitlements management?

The definition and application of security metrics is ongoing. One resource I recommend is Securitymetrics.org, which provides empirical strategies for decision-makers and security practitioners and which includes links to digests, presentations, and handouts from past Metricon Workshops.