Scaling Reliability at Dropbox Our Journey towards a Distributed Ownership
summary
Dropbox shifts reliability into distributed ownership by building a ticketed incident pipeline: memorable drop subs names, four severity levels up to sub-zero, a 24/7 incident manager roster, and postmortems migrated from docs into tickets. Reliability reviews use a discovery/recovery/prevention format and monthly scorecard reports for declared critical systems. Engineering adds continuous testing and runs a Netflix-style chaos monkey (random rack failures twice weekly). Results: database teams hit zero high-severity incidents in Q3, cut MTTR ~25%, and complete half of a legacy DB migration.
AI-written summary · about
chapters (15)
- 0:00Introduction to reliability
- 1:36Defining reliability
- 3:26Early growth and chaos
- 5:59Need for structural changes
- 8:07Core questions for improvement
- 9:52Defining and classifying incidents
- 11:48Incident management process
- 14:25Post-mortem analysis
- 17:03Reliability review meetings
- 20:20Example of incident review
- 23:11Scaling reliability
- 25:22Resiliency and prevention
- 28:03Measuring success with scorecards
- 29:56Results and future outlook
- 30:57Key takeaways for your company
related talks
14mSelective Reliability Engineering: There Is No Single Source...
47mA Deep Specification for Dropbox
28mMaking the Impossible Impossible: Improving Reliability by...
28mTowards a culture where everyone is responsible for downtime
20mUber's Failover Architecture: Reconciling Reliability and Efficiency in Hyperscale...
30mSturdy Clouds Creating Resilient Software Defined Data Centers
41mOnce Upon a Vulnerability: Building Resilience in a Chaotic World
36mDesigning for failure - architecting fault-tolerant system