Accuracy Is Not Enough: Detecting Hidden Risk in CVE Impact Prediction
summary
An applied evaluation showing accuracy hides high-risk failures and proposing confusion-matrix metrics that surface them by design. Uses real Amazon Linux CVE workflows and illustrative cases (a 90% accuracy run that misses a 9.8 OpenSSL exploit) to show how false-negative rate and likelihood ratios reveal dangerous blind spots. Demonstrates simple formulas, prototype dashboards with rolling windows and a Matthews correlation coefficient trust score, and promises an open-source dashboard to operationalize detection.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:07Accuracy fails in vulnerability prediction
- 5:03True positives vs false negatives
- 9:34False positives vs false negatives
- 13:27Case study: accuracy vs impact
- 16:15CVE impact over time
- 19:45Metrics beyond accuracy
- 21:30Disputed CVEs and trust
- 24:57Dashboards for false negatives
- 26:41Metrics in security operations
- 28:08Q&A
related talks
40mOpen Source CVE Monitoring and Management: Cutting Through the Vulnerability Storm
38mCVE — Myth and Reality
15mKeynote
54mTowards Real World Cyber Risk (Panel)
56mCVE Is The Worst Vulnerability Framework (Except For All The Others)
01mEvolution of CVEs and Vulnerability Management
30mCVE Context Matters, but Do All Vulnerabilities Really Matter?
23mSecurity Outgunned: Measuring Software Defined Attack Surface