The Formula for Faster Outage Recovery
summary
Claims a simple formula: outage = time to detect + time to acknowledge + time to repair and shows concrete levers for each. Defines SLO examples like 99% of page loads <300ms or 99% of emails within 15min, recommends 5–10 on‑call owners per critical service, names tools (PagerDuty, OpsGenie, open telemetry, Kubernetes, AWS, CI playbooks) and prescribes measurable practices — dashboards, runbooks, simulated alerts, training — with postmortems to verify whether those mitigations actually shorten outages.
AI-written summary · about
chapters (14)
- 0:00Outage recovery costs
- 1:26Outage recovery formula
- 3:34Service level objectives
- 7:25Detecting with SLOs
- 7:48Acknowledge and on-call
- 10:15Ownership of systems
- 13:24On-call schedule
- 14:28Repair preparation
- 15:33Observability for repairs
- 16:36Engineer training
- 18:21Playbooks and runbooks
- 20:28Incident response stages
- 21:49Learning from outages
- 23:15Closing remarks
related talks
15mMaintaining reliable systems
30mReliability Engineering for Humans
22mDon’t wait for an outage to improve your reliability
29mShrinking the Impact of Production Incidents
20mEvangelizing the SRE mindset
51mShrinking Production Incidents
44mYou build it, you run it
42mYou put out the fire on production, what's next?