# Defining Security Metrics

A metric is a consistent standard for measurement.

Business leaders will ask the following:
- How effective are my security processes?
- Am I better off than I was this time last year?
- How do I compare with my peers?
- Am I spending the right amount of money?
- What are my risk transfer options?

Therefore, Good metrics:
- Incorporate measures of time or money
- Are well understood across the company
- Are well understood across industries and are consistently measured. That means they lend themselves to benchmarking
- Are calculated mechanically and can be gathered in an automated way

Differently put, good metrics:
1. are simple to explain and straightforward to calculate, clear and unambiguous. Their transparency facilitates adoption by management. 
2. are all expressed in terms of time, money, or a measure derived from these. 
3. readily lend themselves to benchmarking.

A good metric should be
- Consistently measured, without subjective criteria – Different people should be able to apply the method to the same data set and come up with equivalent answers.
- Cheap to gather, preferably in an automated way 
- Expressed as a cardinal number or percentage, not with qualitative labels like “high,” “medium,” and “low”
- Expressed using at least one unit of measure, such as “defects,”“hours,” or “dollars” – making it easier to compare.

A good metric should also ideally be Contextually specific: relevant enough (and informative enough) to decision-makers so that they can take action.

“Traffic light” inputs (red-yellow-green) are not metrics at all. They contain neither a unit of measure nor a numerical scale. Traffic lights colors can be used, sparingly, as a presentation strategy to supplement numerical data or draw attention to outliers.

Metrics ought to be computed at a frequency commensurate with the process’s rate of change. For most security processes “often” is better than “sometimes.”

## Reservations toward ISO 17799
My reservations about using the ISO as a metrics framework:
- ISO 17799 portrays information security controls as things to be assessed, selected, and implemented. But it makes almost no practical recommendations about how to manage, monitor, and measure the effectiveness of these controls.
- The recommendations are never firm; interpretation is an exercise left to the reader
- Insufficient attention to measurement

## Reservations toward Annualized Loss Expectancy
Security events are low frequency and high severity. That makes them hard to model.
ALE is security’s spherical cow. It encourages practitioners to think about dollar impact on an aggregate, averaged basis, in spite of the fact that losses do not gravitate to the middle; they cluster on the far edges.
Second, practitioners of ALE suffer from a near-complete inability to reliably estimate probabilities or losses. The same is true for loss estimation.
The third strike against ALE relates to the lack of probability-and-loss data.
The real tragedy of ALE: it’s really just a funny little model that anyone can game for his or her own purposes.