SRE by Influence, Not Authority: How the New York Times Prepares for Large Scale Events
Talkat SREcon19 Europe/Middle East/Africa · on Thursday, 2019-10-03· at 09:45· Track 2: The Liffey A·
summary
Large-scale elections readiness moves from mandates to partnership: over eight weeks (effectively six) more than 20 teams run architecture reviews, implement GCP tooling and Vault for secrets, and adopt a Slack budget bot; 300 applications migrate and 142 are retired. Eleven incident management training sessions teach ~120 engineers a shared runbook and command roles. Production stress testing validates failure modes, one service sustains a 40x traffic spike with minimal user disruption, and registration growth follows.
AI-written summary · about
chapters (19)
- 0:00Start
- 6:02What We Do
- 7:35Product Management Mindset
- 15:04Midterms and Political Climate
- 17:58Execution: Structure
- 19:11Project Design Principles
- 19:49What we needed to do quickly...
- 21:18Architecture Review Template
- 23:34Let engineers raise lurking issues
- 24:11Raise and prioritize things we find
- 25:48Plan Degradation Strategy
- 26:33Understand Roles Roles
- 27:13Focus on bigger picture
- 27:52Emphasize non-engineering bits
- 28:56See What Breaks
- 30:14Little warning for the news
- 30:40Preparations
- 32:16Learning from Results
- 33:21Learning Reviews + Re-test
related talks
43mSRE From Inception to Global Scale
33mChallenges of Starting an SRE Team from Scratch in an Enterprise
35mDeploying SRE Training Best Practices to Production...
26mMission Critical Mindset Building a Culture of Resilience in DevOps Teams
57mBuilding a Modern Security Engineering Organization
28mMaking do: Scaling expertise on teams you can't grow
30mThe Path to SRE
16mSRE in Transition: From Startup to Established Business