Incremental GPU Slicing in Action
Talkat Cloud Native + Kubernetes AI Day NA 2024 · on Tuesday, 2024-11-12· at 16:30· Salt Palace | Level 1 | Grand Ballroom A·
summary
An open‑source controller called insta-slice operator incrementally creates and destroys NVIDIA MIG slices without kernel or device‑plugin API changes by consuming Kubernetes 1.24.6 features, the GPU operator, gated pods, a webhook, daemonset and a placement controller. Demo clusters with two H100 nodes (8 GPUs each) show Jupyter and VM workloads get quarter/half GPU slices, allocations move through creating→created→deleting states, and slices are removed on pod termination. It also exposes an accelerator memory quota to integrate slicing with Kubernetes quotas and Q-based queuing.
AI-written summary · about
chapters (16)
- 0:00GPU utilization challenges
- 2:26Fractional GPU allocation
- 4:34Multi-instance GPU slicing
- 6:20Kubernetes limitations
- 8:06InstaSlice operator
- 11:16InstaSlice components
- 14:27Slice placement policies
- 15:12Live demo setup
- 17:45Jupyter notebook workload
- 19:42Slice lifecycle management
- 20:24VM integration
- 20:44Quotas and queue management
- 22:50Comparison with Kubernetes Dynamic Resource Allocation
- 23:57Future of InstaSlice
- 24:40Conclusion and availability
- 25:02Q&A
related talks
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
38mImproving GPU Utilization using Kubernetes
30mRunning AI Workloads in Containers and Kubernetes
07mProject Lightning Talk: From Static Slices To Elastic GPUs: Dynamic MIG With HAMI