100 Teams, 100 Ways to Fail
summary
Azure SRE describes engagements across 100+ teams, shows concrete fixes (a 40 lines of code chatbot to send group invites, service dashboards wired to live metrics, scripted certificate-rotation work) and reports outcomes: the bot works but trust matters, dashboards immediately convince engineers, automation sometimes stalls; a heavily involved team reclaims SRE work later but reliability and toil metrics improve, and playbook items get reused to reduce 60 people to five during incidents.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:18Distributed systems failures
- 5:54Cultural challenges in failures
- 8:48Incident response detective story
- 17:19Key failure themes
- 17:44SRE as Clark Kent
- 20:17Forgetting to be Superman
- 24:50Building trust and value
- 27:17Avoiding hero culture
- 29:04First engagement with controlled team
- 34:59Failure leading to success
- 36:46Building teams not systems
- 38:09Culture of shared failure
- 39:11Call to action
related talks
29m10 Lessons Learned in 10 Years of SRE
13mSucceeding as the Lone SRE in a Small Team
30mThe Path to SRE
33mChallenges of Starting an SRE Team from Scratch in an Enterprise
31mSRE Theory vs. Practice: A Song of Ice and TireFire
18mThe Good, the Bad and the Ugly: The 3 Learnings of an SRE
45mClearing the Way for SRE in the Enterprise
25mPitching SRE to your Senior Executives