Scaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
Talkat Open Source Summit Europe 2023 · on Wednesday, 2023-09-20· at 14:40· Room 0E-1 (Floor 0), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Kubernetes plus the NVIDIA GPU Operator and device plugin expose GPUs as integer devices so containers can share a card via MIG (up to seven compute slices) or by operator-driven time slicing. MIG physically partitions an A100 into 1/7 compute slices with peculiar allowed sizes (1,2,3,4,7 produce 5/20/40 GB quirks) and often requires a GPU reboot to reconfigure. Time slicing is logical and cheaper to reconfigure but gives variable QoS; Helm, configmaps and node taints/tolerations enforce placement.
AI-written summary · about
chapters (13)
- 0:00Scaling AI workloads
- 3:08Why Kubernetes for GPUs
- 4:11Multi-instance GPU (MIG)
- 7:19Partitioning GPU slices
- 11:36NVIDIA GPU Operator
- 14:26Dedicated GPU nodes
- 16:13GPU sharing limits
- 17:56Sharing GPUs via MIG
- 21:05Time slicing GPUs
- 23:35MIG vs time slicing
- 25:44Optimizing deep learning
- 28:12Summary
- 28:33Question and answer session
related talks
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
30mRunning AI Workloads in Containers and Kubernetes
38mImproving GPU Utilization using Kubernetes
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
39m✨ GPU in My Containers!
05mLightning Talk: Sharing a GPU Among Multiple Containers