Skill Degradation: An Empirical Analysis of 400+ AI‑Generated Security Fixes
summary
Systematic review of 400+ AI-generated patches for real vulnerabilities shows concrete outcomes: many patches either fail to fully remediate or introduce subtle regressions, and reliance on suggestions produces a clear remediation accuracy drop compared with manual fixes. In a secure-coding contest setting a large share of participants produce or accept fixes they cannot explain, demonstrating over-reliance and reduced learning rather than skill transfer. Real examples and measured counts back these claims.
AI-written summary · about
chapters (6)
- 0:00Introduction to study
- 5:31AI-generated security fixes analysis
- 14:25Industry impact and adoption
- 24:21Skill degradation concerns
- 33:05Key findings and conclusions
- 39:19Q&A session
related talks
41mSkill Degradation: An Empirical Analysis of 400+ AI‑Generated Security Fixes
05mThe Asymmetry of AI Security and Physical Hardware Resilience
29mThe vulnerability apocalypse: AI broke the security playbook
09mHow Much Trust Do You Have with LLM-Based Solutions?
43mThe State of AppSec in the Era of AI [Panel]
41mDon’t Make This Mistake: Painful Learnings of Applying AI in Security
57mKeynote: Using LLMs and generative AI to fix software vulnerabilities
56mWhy Auto-Fixing Vulnerable Code Needs More Than Good Intentions