Resources

We have read many and reviewed a few metrics books. Click the book covers above to find out what we think of them.

Metrics papers include:

A few good information security metrics is a well-written, thought-provoking article by Scott Berinato about the need to select and focus on a limited number of valuable metrics. Don’t try to measure anything and everything at random: work out what actually matters first.
ISO/IEC 27004 is an international standard about measurement in support of an Information Security Management System based on ISO/IEC 27001. Metrics are an essential component of the ISMS, although this standard does a mediocre job of explaining how. Thankfully, it is being revised: perhaps the new version will take a more PRAGMATIC line, for example coupling security control-related metrics based on the control objectives specified in ISO/IEC 27002 with security management and governance-related metrics based on the ISMS requirements specified in ISO/IEC 27001.
SP800-55 Rev 1 is the latest version of NIST’s Performance Measurement Guide for Information Security - in our opinion, a much more comprehensive and useful standard than the current version of ISO/IEC 27004. A NIST research paper concerning the earlier version noted: “The universe of possible metrics, based on existing policies and procedures, will be quite large.” Glorious understatement! It continues “Metrics must be prioritized to ensure that the final set selected for initial implementation facilitates improvement of high-priority security control implementation (as defined by an audit or risk assessment). Based on current priorities, use no more than 10-20 metrics at a time.”
Seven myths about security metrics was Gary Hinson’s rant against applying inappropriate criteria to metrics. Mathematically correct metrics are not necessarily quite as worthwhile or valuable as they may appear, in practice.
Metrics: you are what you measure! by Hauzer and Katz from MIT is a seminal paper on the dangers of driving the organization in unintended and perhaps unwelcome or counterproductive directions through the adoption of inappropriate metrics. It’s a salutory lesson in the value of thinking-through metrics and their potential consequences, and an excellent reason to pilot or try-out new metrics before committing to them.
Establishing a Security Metrics Program by SANS MSc students Chris Cain and Erik Couture recommends metrics aligned directly with the SANS top 20 security controls.
A Guide to Effective Security Metrics on the EDUCAUSE wiki suggests that the type of metrics required depends on the state of maturity of the organization’s information security management practices.

Metrics lists and catalogs include:

Debra Herrmann’s Complete Guide to Security and Privacy Metrics details 900 metrics.
The Center for Internet Security’s Consensus Security Metrics is a reasonable, generic shortlist of 21 metrics.
There are a few other metrics shortlists on the Web, and many other sources of inspiration if you remove your IT security blinkers for a moment (read chapter 5).

[As far as we personally are concerned, there are far too many candidate metrics out there, thanks to a surfeit of suggestions on what to measure from well-meaning but naive advisors who cannot possibly know your organization’s specific measurement objectives. We don’t know of any metrics gurus (ourselves included) who waste opportunities to promote their own pet metrics. However, selecting or developing and refining bespoke security metrics using the PRAGMATIC method rather than off-the-peg one-size-fits-all metrics not only leads to a better outcome, but is a fabulous voyage of discovery in its own right. There’s still a lot of uncharted territory out there. Be bold and prosper!]