Remote execution with 100 to 100,000 CPUs
summary
Buildbarn is a remote cache and execution system for Bazel that scales from a few nodes up to 100,000 CPUs by using a sharded front end, mirrored storage, and office read-caches to cut latency. Key operational pieces are a single-process scheduler, workers/runners with optional network isolation, existence caching to avoid repeated blob checks, and sticky scheduling to reuse worker-local outputs. Features include automatic resource-size detection, VFS on-demand file serving (FUSE/NFS), Prometheus/Grafana monitoring, and a completed action logger API for slow-build analysis.
AI-written summary · about
chapters (9)
- 0:00Introduction to Buildbarn
- 1:03Scaling and configuration
- 2:11Office caching strategies
- 3:04Cache optimization
- 4:11Remote execution workflow
- 5:46Error handling
- 6:54Advanced performance features
- 8:26File system and performance
- 9:18Monitoring and conclusion
related talks
21mBuilding Distributed Platforms at Extreme Scale on Fast Changing Data
50mOur Adventure building a massively scalable mixed C++ and Rust remote compiler cloud
29mPost Mortems for 4 Years of Remote Execution
30mBuilding a Distributed Build System at Google Scale
16mBurst Computing: Quick, Sudden, Massively Parallel Processing on Serverless...
43mScalability Intro – Shyam Jeedigunta, Google & Bob Wise, AWS (Any Skill Level)
20mBuilding Apache Heron
35m100 machines do 100s work in 1s