Why Can't Johnny Fix Vulnerabilities: A Usability Evaluation of Static Analysis Tools
summary
Study evaluates four static analyzers including Find Security Bugs, RIPS and Flawfinder plus one commercial tool using heuristic walkthroughs and a follow-up user study. Main findings: propagation graphs become unreadable propagation graphs on real codebases, confidence/severity scores lack scale so numbers like “18 with normal confidence” are meaningless, and co-occurring vulnerabilities get hidden behind tiny UI affordances requiring hover. Authors provide a replication package so tool maintainers can rerun the evaluations and prioritize concrete fixes.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 0:24Static Analysis Tools
- 1:37Research Question
- 2:31Methodology
- 3:17Issues
- 3:48Confidence and Severity
- 4:15Multiple Vulnerabilities
- 4:40Conclusion
related talks
45mFinding Vulnerabilities through Static Analysis and Scripting
43mDo Certain Types Of Developers Or Teams Write More Secure Code?
38mEffective Usage Analysis: Boosting Developer Productivity Via Innovative, Noise-reduced Vulnerability Analysis
33mBattle Tested Application Security
45mEradicating Vulnerability Classes by Shelving SAST and Embracing Secure Defaults and Invariants
01h 03m“Secure Coding Best Practices: Your First Line Is The Last Line Of Defense (2 of 2)”
50mFix The Damned Software
30mTopics of Interest: Effective Usage Analysis: The Shortest Path Between a Developer ...