References

This section of the website points you to additional resources to study security metrics in more depth.

It gets updated from time to time with additional content.

If you know of other relevant, useful resources we might also cite, please let us know.
We are particularly interested in novel approaches, plus those that are themselves measured.

Security metrics/measurement standards

ISO/IEC 27004 is an international standard about measurement in support of an Information Security Management System based on ISO/IEC 27001. Metrics are an essential component of the ISMS, although this standard does a mediocre job of explaining how. Thankfully, it is being revised: perhaps the new version will take a more PRAGMATIC line, for example coupling security control-related metrics based on the control objectives specified in ISO/IEC 27002 with security management and governance-related metrics based on the ISMS requirements specified in ISO/IEC 27001. [Note: the current, second edition substantially improved on the first]
SP800-55 Rev 1 is the latest version of NIST’s Performance Measurement Guide for Information Security. A NIST research paper concerning the earlier version noted: “The universe of possible metrics, based on existing policies and procedures, will be quite large.” Glorious understatement! It continued “Metrics must be prioritized to ensure that the final set selected for initial implementation facilitates improvement of high-priority security control implementation (as defined by an audit or risk assessment). Based on current priorities, use no more than 10-20 metrics at a time.” [Note: revision 2 is in the works, allegedly]

Metrics/measurement methods

ISACA’s COBIT covers a broad range of corporate processes relating to the management of information and IT, along with the associated monitoring, assessment and evaluation activities.
FAIR is a well-respected quantitative method from Jack Jones and the FAIR Institute, used to explore relevant ‘risk factors’ in order to characterize and quantify information risks. FAIR-CAM is a ‘controls analytics model’ based on the FAIR framework, seeking to analyze an organization’s entire suite of information security and related controls as an integral system. According to the FAQ, FAIR-CAM is “primarily intended to enable measuring the value of risk management controls.” So far, it incorporates a few security metrics e.g. “% reduction in the probability that threat actors would choose to act in a way that could result in harm” ... but doesn’t yet explain how that might be quantified and expressed with units of measure. Furthermore, the linkage from metrics through controls and risks to business objectives relating to the protection and exploitation of information is implicit.

Metrics papers

5 Metrics to Measure Progress in Cyber Risk Management Resilience was a video presentation to the FAIR conference by Matt Tolbert, a senior cyber-specialist with the Federal Reserve Bank of Cleveland. He is primarily concerned with significant cyber-attacks threatening stability of the US financial system and hence the US economy as a whole. For example, one of Matt’s suggested metrics for ‘cyber-resiliency preparedness’ is “Completeness of documentation - Records for policies, standards and procedures should be complete in terms of addressing all the [incident] scenarios [including black swans] and should be regularly reviewed, updated, and approved by management.” Strictly speaking, that’s a compound metric concerning several distinct but related aspects, and it’s not easy to determine why that particular metric made it onto Matt’s shortlist (hint: a PRAGMATIC assessment would have been insightful, Matt!).
A few good information security metrics is a well-written, thought-provoking article by Scott Berinato about the need to select and focus on a limited number of valuable metrics. Don’t try to measure anything and everything at random: work out what actually matters first.
Seven myths about security metrics was Gary’s rant against applying inappropriate criteria to metrics. Mathematically correct metrics are not necessarily as valuable as they may appear, in practice.
Metrics: you are what you measure! by Hauzer and Katz from MIT is a seminal paper on the dangers of driving the organization in unintended and perhaps unwelcome or counterproductive directions through the adoption of inappropriate metrics. It is a salutory lesson in the value of thinking-through metrics and their potential consequences, and an excellent reason to pilot or try-out new metrics - and determine their effectiveness - before committing to using them permanently.
Establishing a Security Metrics Program by SANS MSc students Chris Cain and Erik Couture recommends metrics aligned directly with the SANS top 20 security controls.
A Guide to Effective Security Metrics on the EDUCAUSE wiki suggests that the type of metrics required relates to the state of maturity of the organization’s information security management practices - in other words, security metrics and the associated measurement processes naturally evolve and mature over time.

Metrics lists and catalogs

Debra Herrmann’s Complete Guide to Security and Privacy Metrics details some 900 metrics!
The Center for Internet Security’s Consensus Security Metrics is a reasonable, generic shortlist of 21 metrics.
The Cloud Security Alliance’s 34 continuous cloud security audit metrics are worth considering.
ISO/IEC 27004 has a few rather lame metrics examples, whereas the main body of the standard is strong.
There are a few other metrics lists on the Web, and many other sources of inspiration if you remove your IT or cyber-security blinkers for a moment (read chapter 5 of our book!).