What is new in security metrics is the growing understanding that many of our traditional efforts at measurement are unsatisfactory, writes Lance Hayden in ‘IT Security Metrics: A practical framework for measuring security & protecting data’ (www.tatamcgrawhill.com). Such efforts do not give us the information we really need to support decisions and articulate the value of security activities; and they are not adequate for the changing security landscape of more subtle threats and increased accountability and scrutiny, he notes.
Immediate concerns
A common challenge that the book highlights is the tendency to localise metrics to one’s immediate concerns. “We tend to measure only those things that we deal with regularly, and eventually we decide those are the only measurements that matter,” the author cautions.
As analogy, he describes how every morning he makes coffee, carefully measuring several scoops of ground coffee and several cups of water into a French press as part of his daily caffeine ingestion ritual. “I care about these measurements because they directly affect my morning. I don’t think about how these measurements are related to other metrics, such as the proper acidity and nitrogen levels for growing coffee or the optimal temperatures and durations for roasting it. I depend on others for these measurements (although if they are incompetently performed, I find another source for my coffee).”
Beyond local context
Reminding, therefore, that metrics – be they for coffee or IT security – involve many local and tactical efforts which become increasingly interdependent and strategic as they begin to affect larger systems, Hayden calls for a greater understanding of measurements outside of one’s local context so as to make better decisions.
For instance, you may not measure security beyond analysing the contents of your firewall logs, but if you do not understand how others measure security or other business values, you will be less able to use your data for making good decisions with a big-picture approach, he explains. “As security becomes more complex and pervasive, and security professionals are held responsible not only for protecting company assets but also for contributing to its financial and competitive success, information about how IT security operates will be more globally and strategically relevant.”
A journey rather than a destination
The author rues that people setting up a security metrics programme generally make the mistake of focusing too much on the metrics themselves, as if metrics were a destination rather than a journey. He instructs that once you have established a security metrics programme you must ask yourself how the results of the programme have improved your understanding of your security systems and processes.
Underlining further that understanding is not diagnostics, the author gives the example of how knowing year after year that some percentage of your users’ passwords are easily cracked or that the ratio of vulnerable to secure Internet-facing hosts has not dropped below 1-in-4 reduces some of the uncertainty regarding your IT security effectiveness, but if the information has not enabled you to improve that effectiveness, something is missing from the programme.
Again, even if the security has improved, if that is all you know and you cannot say why the improvement occurred, your metrics are not giving you any more value than if you were struggling over why your security was getting worse, Hayden argues. “Metrics are conceptual data repositories – they define and standardise information. Metrics do not organise that information into knowledge, any more than well-defined word entries will transform a dictionary into literature. Only people can accomplish these things.”
Mountains of data
The intro chapter devotes a major section to the point that measurement is an activity; that it is the activity of making observations and collecting data in an effort to gain practical insight into whatever we are attempting to understand.
Hayden clarifies that collecting metrics data for the sake of metrics data is not measurement unless the purpose of the activity is to mine historical data for interesting patterns as a research exercise.
Bemoaning the usual phenomenon of collecting a lot of data in the name of security metrics, he counsels that a small set of data, understood well and applied regularly, is much more valuable than a mountain of data left untouched on shelves or hard drives and gathering real or virtual dust!
Measurement without analysis
A valuable warning in the book is that measurement without analysis and action wastes time and money and contributes to uncertainty and risk rather than reducing them. Knowing about a problem and not having acted upon it, leading to a security breach, however, could actually end up more damaging than the ignorance that existed before you ever gathered the data, observes Hayden.
Lamenting that many security managers do not consider the idea that the data they collect becomes a matter of corporate record and possibly subject to e-discovery, he alerts that unused metrics data can simply add insult to injury. “You still get hacked, but you also lose the resulting lawsuit because you ‘knew’ you could get hacked based on your security metrics data. This is an important consideration for security metrics that is only beginning to be discussed in our industry.”
The moral, though, is not to strive to know as little as possible about how your security is functioning, but to approach IT security based on a sound strategy, ‘with the same eye toward risks, costs, and benefits’ as in the case of any other business process.
A compelling read that can prepare you for the many IT security risks enterprises face.
**
Tailpiece
“We embedded so many security devices into our watchdog…”
“That it stopped barking?”
“No, it has become a robot!”
**
BookPeek.blogspot.com
