Declarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
summary
Explains why MIG, MPS and time‑slicing trade isolation, latency and flexibility and shows a software alternative: Hammy (HAMI) CNCF sandbox that intercepts CUDA runtime calls via hammy‑core, exposes fractional GPUs via GPU memory and GPU cores (GPUme/GPUcores), and enforces memory and fault isolation in software. Demo schedules three pods each requesting 30% cores + 4GB on one GPU concurrently and a fourth goes pending. Discusses DRA (Dynamic Resource Allocation) (claim-based API, decouples pod/device lifecycles; GA targeted for Kubernetes 1.34, beta in 1.31/1.32).
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:23GPU limits in Kubernetes
- 3:51Multi-instance GPU strategies
- 9:52MPS and partitioning
- 13:03Introducing HAMI
- 21:58HAMI architecture
- 23:43Live HAMI demo
- 26:55Dynamic GPU allocation
- 30:46HAMI vs DRA
- 31:50Key takeaways
- 32:12Q&A
related talks
53mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
05mHami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
09mLightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
30mRunning AI Workloads in Containers and Kubernetes
31mAI, CERN, and the Quest for GPU Custody: How CERN Leverages DRA for Efficient GPU Sharing
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
04mLightning Talk: Dynamic GPU Fraction and Sharing With Cloud Native Principle