Multitenancy and Fairness at Scale with Kueue: A Case Study
Talkat Cloud Native + Kubernetes AI Day NA 2024 · on Tuesday, 2024-11-12· at 10:40· Salt Palace | Level 1 | Grand Ballroom A·
summary
Kueue augments Kubernetes scheduler and ClusterAutoscaler to run batch/ML workloads with fair sharing (v0.7) and reclaimable quotas via preemption, claiming higher utilization while letting teams borrow unused capacity; example: a 24‑GPU cluster with three queues each nominally at 8 GPUs where borrowers show counts like 3 and 2 and the lowest share runs first, highest share gets preempted. Recent additions: hierarchical cohorts (v0.9), provisionRequest Beta for bulk scale‑up, qctl CLI, gang‑aware scheduling, RDMA/zone‑aware placement and preemption throughput improvements.
AI-written summary · about
chapters (8)
- 0:00Kueue introduction
- 2:08Cluster resource problem
- 3:54Fair sharing in Kueue
- 6:47Hierarchical cohorts
- 10:38ML platforms with Kueue
- 14:31ML workload features
- 20:33Kueue 0.9 highlights
- 23:25Q&A
related talks
33mBatch Systems in Production with Kueue: Multi-Tenancy and Fungibility
10mLightning Talk: Resource Fairness and Utilization for Heterogeneous Batch/M...
29mBuilding a Batch System for the Cloud with Kueue
35mKueue: A Kubernetes-native Job Queueing
24mAccelerate Your AI/ML Workloads With Topology-Aware Scheduling in Kueue
36mCapacity Scheduling for Elastic Resource Sharing in Kubernetes
10mFair Scheduling for Deep Learning Workloads in Kubernetes
30mTopology-Aware Scheduling for AI Training & Inference With Kueue