Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning
Talkat 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21) · 2021-07-14..07-16·
summary
Pollux builds a co-adaptive scheduler that jointly models system throughput and statistical efficiency, introduces the goodput metric (their product), and dynamically adjusts GPU counts, batch size, learning rate and gradient accumulation. It fits performance models online, searches allocations with a genetic meta-heuristic, and is implemented as open-source code. On real jobs and trace-driven simulation (160 jobs over eight hours) it reduces average completion time by 37–50% versus state-of-the-art, tested on a 64-GPU cluster.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:26Deep Learning Training in Shared Clusters
- 0:54Example Shared-Cluster DL Training Workflow
- 1:36Pollux: Co-adaptive Cluster Scheduler for DL
- 2:16Outline
- 3:06System Throughput and Impact of Batch Size
- 3:41Statistical Efficiency and Impact of Batch Size
- 4:31illustration of Overall Training Performance
- 5:03Implications for Cluster Scheduling
- 5:49Pollux Cluster Scheduler
- 6:21Key Idea: Goodput, not Throughput
- 6:59Modeling System Throughput
- 8:00Modeling Statistical Efficiency
- 10:05Optimizing Cluster-Wide Allocations
- 11:11Evaluation of Pollux
- 12:36Cluster-Wide Statistical Efficiency
- 13:25More Experiments in our Paper!
- 13:41Conclusion
related talks
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...
05mProject Lightning Talk: Scheduling AI Workload Among Multiple Clusters
15mUSHER: Holistic Interference Avoidance for Resource Optimized ML Inference
24mTiresias: A GPU Cluster Manager for Distributed Deep Learning
22mBeyond Single-Cluster Limits: Scaling GPU Workloads Across Kubernetes With…
36mAccelerating High-Performance Machine Learning at Scale i...
36mAccelerating AI training fleets with sched_ext
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective