Toil and Trouble
summary
Defines toil using Google SRE criteria (repetitive, manual, no enduring value, tactical) and shows concrete tooling and outcomes: platform runs on BOSH and deploy pipeline uses Concourse CI; team tracks recurrence with a toil snake of stickies and prioritizes with a 2x2 of impact vs frequency. Two solved cases: abusive workloads progressed from detection/notification to automated story creation and planned auto-suspension (not yet enabled), and support flows moved from Slack acknowledgements to auto story creation, follow-up surveys and eventual self‑service. Practices include rotations, retros, and SLO workshops.
AI-written summary · about
chapters (12)
- 0:00Start
- 3:40Toil Provides no Enduring Value
- 4:13Tactical Work
- 10:28Toil Snake
- 11:53Prioritizing and Filtering for Toil
- 12:37Filtering for Toil
- 15:46Incident Handling and Reporting
- 18:15Automatic Suspension
- 20:52Act 3
- 23:19Command and Control Model
- 26:20Product Working Groups
- 27:03References and Resources
related talks
34mDouble, Double Toil and Trouble; SRE Transform...
26mDouble, Double, Toil and Trouble!
38mWhy Automating Everything Adds to Your Toil
22mSRE and Risk management: a love story
37mImprove Your Automation to Reduce Toil
32mTales from the DevOps Transformation Trenches
31mOperations Reform: Tom Sawyer-ing Your Way to Operational Excellence
43mWhere SRE Can Go Wrong