You put out the fire on production, what’s next?
summary
After production recovery, run a postmortem within 1–3 days for incidents that meet SRE criteria (user-visible downtime beyond SLA, data loss/corruption, monitoring failure or customer-reported detection). Build a factual timeline, root causes, action items: use fishbone/tree mapping rather than only Five Whys, record who did what without blaming, and make action items small, owned, and automated where possible. Invite on-call, ops/db admins, product and support for context, set deadlines and verification so fixes aren’t buried, and rerun drills or simulations to confirm the fixes work.
AI-written summary · about
chapters (13)
- 0:00Introduction to postmortems
- 1:07Postmortem analysis basics
- 3:17Immediate recovery steps
- 5:45When to run postmortems
- 7:52Blameless culture and templates
- 10:26Collecting evidence and timeline
- 11:31Building the incident timeline
- 12:56Root cause analysis methods
- 15:46Defining actionable items
- 16:30Who to invite to postmortems
- 17:56What to emphasize and avoid
- 19:47Common postmortem pitfalls
- 20:50Verifying and sharing results
related talks
42mYou put out the fire on production, what's next?
22mAfter the dust has settled: learning from incidents
29mShrinking the Impact of Production Incidents
36mSurviving in Production: Incident Management and Learning
40mProduction Detectives: Solve the Case Before the Crash
27mPostmortem Action Items: Plan the Work and Work the Plan
34mIncidents are new normal
15mPragmatic Incident Response: Lessons Learned from Failures