More Security Metrics

Although I wrote about Security Metrics: Replacing Fear, Uncertainty and Doubt by Andrew Jaquith earlier, a single post doesn't do this important topic justice. The key theme as expressed by Jaquith is

...information security is one of the few management disciplines that has yet to submit itself to serious analytic scrutiny.

This lack of analytic scrutiny in the form of security metrics makes risk management especially difficult for executive understanding and guidance, especially when discussing the necessary level of investment required. Executives ideally want their security and compliance metrics to answer the following questions:

• How effective are my security processes?
• Am I better off than I was this time last year?
• How do I compare with my peers?
• Am I spending the right amount of money?
• What are my risk transfer options?

As previously discussed, most functions within an enterprise -- HR, finance, manufacturing, supply chain, call center, e-commerce and operations -- have the ability to measure their performance by tracking key metrics, and comparing with other companies in a peer group. Such metrics share the characteristics of being simple to explain, readily lending themselves to benchmarking, and being consistently and automatically collected.

Without such metrics, we're doomed to reactive rather than proactive risk management. Or, as Jaquith calls it, we're on the hamster wheel of pain:

Here are Jaquith's suggested questions for management when measuring audit and compliance processes and their related investments:

1. How much time and effort are security staff spending on audit-related activities? (Metrics: # regulatory audits completed, time/cost of audit activities)
2. Have audits uncovered serious weaknesses in existing controls? (Metrics: % security compliance reviews with material weaknesses, % key external requirements compliant per external audit)
3. How much time and effort are security staff spending fixing problems uncovered by audits? (Metrics: # pending deficiencies and estimated time/cost to complete, time/cost spent on remediation activities)
4. Have audit activities uncovered problems with controls that would affect customer trust or privacy? (Metric: # pending customer-related deficiencies and estimated time/cost to complete)

Only by employing security metrics and submitting to serious analytic scrutiny can an enterprise get security and compliance risk management off of the hamster wheel of pain and onto a level playing field with other disciplines.