Exploring the Unintended Consequences of Automation in Software
summary
Research mines the VOID dataset of 10,000+ public incident reports, filters ~450 cases to ~200 for close thematic analysis, and finds automation often multiplies problems: examples include retry storms, lost logs and surprising cross-component interference; Night Capital loses $460M in 20 minutes; a bio analogy notes ~500,000 excess deaths from diclofenac-driven vulture collapse as a cautionary precedent. Key quantitative takeaway: 75% of incidents require human intervention and automation is rarely the final fix.
AI-written summary · about
chapters (8)
- 0:00The vulture case study
- 4:09Introduction to the VOID
- 5:48Rethinking automation
- 9:27Theories of automation
- 14:13The ironies of automation
- 18:34Thematic analysis results
- 23:42Key findings from incidents
- 25:29Moving toward joint systems
related talks
34mExploring the Unintended Consequences of Automation in Software
55mThe Troubles of Automating “all the Things”
42mWhen /bin/sh Attacks: Revisiting "Automate All the Things"
38mWhy Automating Everything Adds to Your Toil
32mWhat Not to Automate
05mA Case Against Automation
41mWhen /bin/sh Attacks: Revisiting "Automate All the Things"
39mLessons Learned in Automation