 Security Metrics
A Beginner’s Guide
Author: Caroline Wong
Publisher: McGraw Hill, 2012
Pages: 397
ISBN: 978-0-07-174400-3
Buy it from: Amazon
With practical tips and useful notes throughout, Caroline’s book is admirably easy to read, although it is not entirely clear whether it is aimed at beginners to security metrics, or beginners to security, or perhaps both.
The book is strong on project management metrics, taking account of social factors when presenting metrics, and technical design of an enterprise wide XML metrics database system. There are plenty of suggestions on prioritizing security activities.
The book’s focus on automated data collection and aggregation for large enterprises over-emphasizes operational IT security metrics over strategic, non-technical and ‘soft’ measures – a bias shared by many security professionals from IT/technical backgrounds. Although Caroline appreciates the need to measure security processes as well as technical controls, the book feels unbalanced.
Despite the promising heading “Decide what to measure”, section III (chapters 5 and 6) offers little advice on how to identify what needs to be measured and select worthwhile security metrics. Chapter 5 “Identify core competencies” concerns IT security competencies, outsourcing of security and the performance of technical security activities such as changing firewall rules. The few metrics suggested here are labeled ‘quantitative’ (e.g. “Percentage of patches deployed within the timeframe specified in the information security group’s service level agreement”) or ‘qualitative’ (e.g. “Which business units receive network vulnerability scan reports from the information security team?”).
We are puzzled at Caroline’s interpretation of ‘qualitative metric’: the examples she offers are mostly questions that would generate lists of items (such as “Who has access to the system?”) or binary answers (such as “Does this technology integrate with third-party partners or providers?”) rather than the kinds of numeric measurements that are generally considered to be metrics. Questions of this nature are more commonly associated with IT compliance audit checklists than information security metrics. Having said that, they are a kind of assurance measure.
Chapter 6 “Identify targets” talks about measuring security things that are important (e.g. compliance and risk), broken (security processes or technologies that might be improved), basic (immature parts of the information security program), worth discussing (issues of interest beyond the security team) or new (costs and functional requirements for new technologies). That’s not a bad generic shopping list of potential security metrics for someone who doesn’t have a clue where to start, although we would recommend a more systematic and structured analytical approach that is much more clearly focused on what the organization expects to achieve through information security. The GQM (Goal, Question, Metric) approach ably described by Lance Hayden is - in our opinion - more likely to generate security metrics that support the business.
|
