You Can't Measure Risk
summary
Argues that most security risks are not truly measurable and that measurement often confuses probability with loss, using CVSS and FAIR as concrete examples: CVSS produces many combinatorial scores but 15 years of data yield ~30 common values, and actuarial annualized loss expectancy treats history, not future risk. Recommends classifying incidents/hygiene/programs, prioritizing the easiest fix among mid‑tier hazards, and treating measurement as communication to drive change rather than an exact science.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:02Risk via gambling game
- 3:50Defining risk and forms
- 8:04Adversaries and environmental risks
- 10:07Risk scenario components
- 11:10Why risk measurement is hard
- 13:37Risk identification and triage
- 17:31Risk severity and likelihood
- 20:20Qualitative risk limits
- 24:31CVSS flaws in risk measurement
- 26:16Fermi problems in risk assessment
- 30:07Actuarial model pitfalls
- 33:38FAIR model challenges
- 36:26Aggregating risk impacts
- 39:57Illusion of measurable risk
- 41:22Human factors in perceived risk
- 44:11Posture and maturity metrics
- 46:37Risk communication frameworks
- 49:47Key takeaways
related talks
21mRisk Is Not a Hammer, and Most Hazards Aren't Nails
59mKeynote: Measure Security Effectively
46mBusinesses Run On Risk And Debt: Why Communicating Security Risk Is Hard
45mThe Business Value of Security
44mOpening Keynote: Prove It! Confronting Security With Data.
26mThe Risky Business of Risk Illiteracy (Sean Juroviesky)
36mNot Your Mama's Risk Assessment
39mFrom Metric to Risk Assessment — How Supplied Data Can Influence Outcomes