Incident Archaeology: Extracting Value from Paperwork and Narratives
summary
A year-long review of incident artifacts (Jira, PagerDuty, Google Sheets, Jupyter) analyzes about ~130 incidents/year, uses roughly 40 analyst hours/year, and involves data scientists to validate charts. Findings: postmortems rise from 55% to 62% after tooling and training, 80% during business hours, only 30% tie to local code changes, MTTR proves misleading in aggregate, and teams cannot reliably define incident start/end—so lightweight async writeups and targeted remediation reporting deliver actionable, repeatable insights.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:59A word on sharing in public
- 2:39Breadth, not depth
- 3:29Are incidents just paperwork?
- 4:12So what's the value of the paperwork process
- 6:06Start Digging!
- 6:48Our first hypothesis
- 9:06Complexity is so simple
- 10:22Data science enters the chat
- 11:39Disappointing Findings
- 12:40Productivity Impact
- 13:20People want answers!
- 13:56How to do Incident Archaeology
related talks
37mLearning from Incidents at Scale; Actually Doing Cross-Incident Analysis
44mProduction Incident Evaluation as Cost of Quality
34mIncidents Happen
37mIncident Management Metrics That Matter
20mData Incidents - When things go wrong
37mIncidents are a new normal
41mAre We Getting Better Yet? Progress Toward Safer Operations
39mNavigating the Chaos: A Holistic Approach to Incident Management