Replacing Fear, Uncertainty, and Doubt
Author: Andrew Jaquith
Publisher: Addison Wesley, 2007
Price: ~US$37 from Amazon
Security Metrics is promoted as a comprehensive guide to security metrics best practices. The coverage is certainly broad, ranging from the mathematical considerations underpinning metrics to their use in measuring technical security elements such as antivirus and application security, and managing security programs. However, most of the metrics are IT/technical metrics, measuring security technologies, technical processes and their inputs: the broader aspects of information security as opposed to IT security are not so well represented.
The final chapter on designing security versions of Kaplan and Norton’s Balanced Scorecard is as challenging as it is helpful. The original Balanced Scorecard used a handful of metrics on each of four perspectives (financial, customer, internal business processes, and learning and growth) as a decision support tool for corporate performance management. Andrew proposes modifications of the perspectives to suit security management (e.g. changing the customer perspective from that of customers of the business to internal customers of the security function), and suggests a selection of metrics for the altered perspectives. Although he is reasonably explicit about the proposed changes, readers are encouraged to consider and adapt the suggestions to suit their circumstances rather than simply follow a generic menu.
Andrew is a prolific, highly-regarded and influential writer, widely acknowledged as a guru in the field of security metrics. He writes in a forthright style that appears to discount, perhaps even discredit alternative approaches and opinions that he clearly disagrees with, and as such he has at times led the field down a narrow, somewhat introspective path. His strident descriptions of good metrics, bad metrics and non-metrics, for example, have perhaps unwittingly constrained the professional dialog to number theory and statistics. According to Andrew, metrics must be expressed as a cardinal number or percentage, not with qualitative labels like “high”, “medium” and “low”, and expressed using at least one unit of measure, such as “defects”, “hours” or “dollars”. We consciously favor a more practical though less academically rigorous approach reflecting the way metrics are actually being used to measure and manage information security in organizations around the globe. Andrew is quite right to point out that it is mathematically unsound to calculate compound security risk scores using simple arithmetic based on “High = 3, Medium = 2 and Low = 1” … but in the absence of a better approach to assessing security risks, such analysis arguably serves a valuable practical purpose. Traffic-light reports are facile … but as a means of focusing management attention on serious security matters, they clearly have a minor but legitimate rôle in business.