Representativeness in the Benchmark for Vulnerability Analysis Tools (B-VAT)
Talkat 13th USENIX Workshop on Cyber Security Experimentation and Test (CSET '20) · on Monday, 2020-08-10·
summary
Claim: current vulnerability corpora misrepresent real-world weakness proportions and that a benchmark must mirror those proportions. Uses CVE (≈75,000 community-accepted instances) correlated with ≈800 CWE types, shows one pillar (CWE 664) covers ~45% of 2014–2019 CVEs, and compares distributions in Juliet, CGC, OWASP Benchmark and Stone Soup. Builds B-VAT by taking a stratified random sample to preserve proportions, reducing a 55,000-case pool 55,000 to ~2,000 while aiming to evaluate tools like KLEE, Mayhem, Angr, Z3 on representative mixes.
AI-written summary · about
chapters (7)
related talks
31mValidating Vulnerability Analysis with Statistical Analysis of Metadata
56mCVE Is The Worst Vulnerability Framework (Except For All The Others)
58mWhy Can't We All Just Get Along? Bridging the Gap in Vulnerability Prioritization Standards
57mVulnerability Root Cause Mapping with CWE
54mTowards Real World Cyber Risk (Panel)
32mUnderstanding Exploitability with VEX, EPSS, and Other Standard Frameworks
58mNothing to Risk but Risk Itself: Expanding Vulnerability Risk with Internet-Scale Data
01mEvolution of CVEs and Vulnerability Management