The Bitter and the Sweet of Running a Planet-Scale Build &...
summary
Runs a centralized Build & CI stack for 100,000+ engineers, sustaining 98% cache hit rate and handling hundreds of petabytes/day via Rabin chunk deduplication and lossless compression; uses local, remote and P2P caching plus a virtual file system and shared build-graph to reuse work. Growth is double-digit year-over-year, countered half by software optimizations and linear-programming placement; 24/7 ops run from two sites with 12‑hour shifts. Risks: cache regressions below 97%, stale/corrupt cache, circular dependencies and faulty build storms.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 2:26Build and CI stack basics
- 6:24Life of a build
- 7:52Build stack traits
- 10:00Resource management at scale
- 11:06Stack dimensions
- 13:37Caching layers
- 14:59Artifact optimization
- 15:59Build execution evolution
- 17:26Backend optimizations
- 18:30Team’s build stack
- 19:15Team operations
- 20:20Simplifying systems
- 21:03Service level objectives
- 22:49Build system challenges
- 26:44Resources and closing
related talks
28mGrowing Pains - What happens when your product gets BIG?
47mHow to win in software development with tighter feedback loops
47mDeploying Like a Boss: From 2 Servers to 20,000
20mMaximize development efficiency with expert CI/CD strategies
20mScaling up a Super-App
46mThe Software Factory
27mImproving the Usability of Remote Caching
50mOur Adventure building a massively scalable mixed C++ and Rust remote compiler cloud