✨ GPU in My Containers!
summary
How to provide GPUs to containerized applications on Kubernetes using the NVIDIA GPU Operator: installation, configuration, host interactions and kernel module management. It covers resource-sharing modes (time-slicing, MIG), cost-efficient GPU use, and supporting add-ons like node-feature-discovery, presented as a step-by-step guide for running LLM workloads on private clusters.
AI-written summary · about
chapters (31)
- 0:00GPU and containers intro
- 2:33Speaker intro
- 3:15Python devs and GPU
- 4:21First demo setup
- 5:04Chocolatine deployment
- 7:15GPU config issues
- 8:39Nvidia GPU Operator role
- 10:05GPU Operator components
- 12:32Nvidia driver install
- 13:12Container Toolkit runtime
- 14:15K8s GPU plugin
- 14:56GPU Feature Discovery
- 17:23GPU components recap
- 17:45Verify component install
- 18:49Chocolatine finally running
- 20:36Chocolatine image build
- 21:21Pecheibou app deploy
- 22:27GPU sharing intro
- 24:09Time Slicing
- 24:53MPS for sharing
- 25:56MIG for sharing
- 28:23Enable MIG on node
- 30:15MIG profile config
- 31:03Pecheibou app with MIG
- 32:07Sanglier image with MIG
- 33:55GPU metrics with DCGM
- 34:16GPU metrics visualization
- 35:23Deploy to other clusters
- 36:49GPU Operator on Skyway
- 37:11Conclusion
- 37:32Q&A
related talks
30mRunning AI Workloads in Containers and Kubernetes
30mExplain How Kubernetes Works With GPU Like I’m 5
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
38mImproving GPU Utilization using Kubernetes
26mAutomating GPU Infrastructure for Kubernetes
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
32mEverything you Need to Know about using GPUs with Kubernetes