Understanding the Reproducibility of Crowd-reported Security Vulnerabilities
summary
Manual reproduction of 368 crowd-reported memory-error vulnerabilities uses 3600 man-hours and finds very low reproducibility: a 256-case subset from SecurityFocus yields 12% success rate from single sources, improved modestly by pooling the top five websites. Reproducers apply default environment settings, Docker and manual debugging, prioritize trigger inputs, and recover missing details by correlating similar reports. A public dataset with VM images and ~300 reproducible cases is released on GitHub.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:14Massive Crowd-reported Vulnerabilities Over Time
- 2:11Vulnerability Reproduction Can Be Challenging
- 2:45Consequences of Poor Reproducibility
- 4:32Roadmap
- 6:57The Analyst Team
- 10:04Reproduction Experiment: Controlled Information Source
- 11:38Vulnerability Is Difficult to Reproduce
- 13:24Key Factors Make Reproduction Difficult
- 14:39Useful Tips for Information Recovery
- 16:22Our Ideas of Making Vulnerability Reproduction Easier
- 18:17Conclusion
- 19:03Data Sharing
related talks
18mStack Overflow Meets Replication: Security Research Amid Evolving...
48mHow to report a vulnerability: Responsible Disclosure for Developers
01mEvolution of CVEs and Vulnerability Management
30mThe Myth of the Meteoric Rise in Vulnerabilities
43mHuman factors that influence secure software development
27mThrough the Looking Glass: How Open Source Projects See Vulnerability Disclosure
56mHow Building a Better Hacker Accidentally Built a Better Defender
23mDemystifying Unreproducible Builds: What, Why, and How?