Engineering for the Long Game
summary
A roadmap for making a large distributed service sustainable: aim for exponential scaling with sublinear team growth by offloading machine ops to orchestration (Borg or cloud equivalents like AWS Auto Scaling and ELB), standardize configs and add a tiny HTTP status endpoint on each box for quick triage, require timeouts and exponential backoff with jitter, prefer boring consolidated infra but pick the right moment, build shared services (an image host took two years and cost ~5% per team but succeeded), and refactor incrementally (webserver moved to multithread over two years).
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:06Google in 2004
- 2:06Planning for the long game
- 2:52Complexity
- 3:14Team
- 4:12Systems
- 4:43Scaling
- 6:54Standardization
- 8:17Maintenance
- 11:07Consolidation
- 12:32Shared Systems
- 14:58Start Over
- 15:29Early Investment
- 17:05Biggest Customer First
- 17:42Final Thoughts
related talks
01h 02mTools for simplified large scale systems
32mScaling Done Right
49mHypergrowth scaling made simple
42mStart As You Mean To Go On
46mDigital scale-up architectural patterns
34mTen-year Journey to 10,000 Production Machines
44mThe Art of Architecting for Scale
14mCreating tools to cross machines and networks