Operationalizing ML Training Infra at Meta Scale
summary
Meta re-architects AI infrastructure end-to-end: custom GPU-heavy servers with layered lossless networking, PyTorch-based distributed training, and procurement tied to P90/P95 commitments. Operational moves include a new failure-detection suite, a wasted-time metric for job restarts, on-demand checkpointing before preemption, batch maintenance to avoid repeated restarts, and single-node fault tolerance in data readers. Result: training throughput improves ~40x and the fleet runs models up to 12 trillion parameters.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:31Use Cases @ Meta
- 1:16ML Execution Stages
- 3:31Al Hardware is more Complex
- 6:20Server Lifecycle
- 6:36Supply & Assembly
- 7:52System Integration
- 9:04Quality Assurance
- 10:44Production Maintenance
- 12:41Al Software has unique properties
- 15:32Scheduling
- 17:21Job Execution
related talks
25mTraining Through Failures: How Meta Keeps 100k-GPU Jobs Alive With Open-Sou...
32mOptimizing Machine Learning Training Infrastructure: A Governance Approach
30mEssential Ingredients in Scaling Organizations for ML
27mHow to make ML teams 80% more productive.
13mMLOps: From AI Research to Production-Ready Systems
28mScaling AI Infrastructure
41mFrom Prototype to Production Building a Scalable MLOps Platform from a Reference Arch
33mBuilding Resilient (ML) Pipelines for MLOps