Beware of Fragmentation: Scheduling GPU-Sharing Workloads with Fragmentation...
summary
Paper defines a statistical fragmentation metric that predicts expected unallocatable GPU capacity given task distributions, implements a scheduler called Fragmentation Gradient Descent as a pluggable Kubernetes scheduler, and evaluates it with Alibaba production traces on an emulated cluster of ~6,200 GPUs. FGD computes hypothetical placements in parallel (~100 ms per decision), packs tasks onto ~250 fewer nodes, and reduces unallocated GPUs by 33–49% (recovering ~150–290 GPUs) compared with packing baselines.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 0:27Background
- 3:48Existing Approach
- 7:33Fragmentation
- 15:57Fermentation Gradient Descent
- 19:15Evolution Results
- 20:54Conclusion
related talks
33mTurning Fragmented GPU Clusters Into One Elastic Compute Pool
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
43mThe GPU Utilization Problem: What’s Going Wrong and How to Solve It
29mSharing is Caring – Fractional GPU Allocations With MetaGPU Device Plugin
28mGPU Reservations: Maximizing Utilization and Fairness Across Teams
38mImproving GPU Utilization using Kubernetes
21mExperience Operating Large GPU Clusters at Organizational Scale
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes