When will my ML Job finish? Toward providing Completion Time Estimates through...
Talkat 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24) · 2024-07-10..07-12·
summary
Claims a system that gives reliable completion-time estimates for ML jobs by configuring Weighted-Fair-Queueing so preemption is bounded and predictable. Uses a simulation-aided search to pick queue counts, exponential weights and an efficiency cap K that limits each job’s max GPUs. Implements PCS and evaluates on a 16‑GPU AutoML testbed plus large simulations: achieves 30% less prediction error while staying within 5% of the best completion time and cuts error by 80% for 95% of jobs.
AI-written summary · about
chapters (13)
- 0:00Completion time predictions in ML
- 1:07Predicting job completion in GPU clusters
- 2:10Preemptive vs non-preemptive scheduling
- 5:04Predictability Centric Scheduling overview
- 6:11Motivation for ML predictions
- 7:57Challenges in ML completion times
- 8:40Weighted Fair Queues for predictability
- 10:06Optimizing Weighted Fair Queues for ML
- 13:15Trade-off navigation parameters
- 14:41Simulation-based search strategy
- 16:02Operator objectives interface
- 17:08Evaluation results and findings
- 18:51Summary and conclusion
related talks
15mMLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU...
25mSupercharging Pipeline Efficiency with ML Performance Prediction
16mIntelligent Resource Scheduling for Co-located Latency-critical Services: A Multi-Model
44mML in Production
12mUse ML to Predict Delivery Time and Improve Code Reliability
22mReal-Time GPU Job Scheduling Latency Prediction in Multi-Cluster Kubernetes
35mOn Machine Learning for Prediction and Prioritization in the Allocation of Scarce Societal Resources
13mPreview: ML Training