Hami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
summary
GPU partitioning middleware that lets multiple containers safely share a single card by injecting a custom CUDA library to enforce hard resource isolation and device memory caps (example: 10GB memory, 30% compute), allocates slices via standard resource limits instead of custom claims, installs via helm and labels nodes, integrates with Volcano for scheduling, requires NVIDIA driver >=440 and Kubernetes >=1.18 (HAMI DI needs >=1.34), supports 10+ vendors and is deployed by 100+ enterprises on 100K+ nodes for higher GPU utilization.
AI-written summary · about
related talks
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
53mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
34mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
07mProject Lightning Talk: From Static Slices To Elastic GPUs: Dynamic MIG With HAMI
30mRunning AI Workloads in Containers and Kubernetes
28mSimplifying AI for Edge Compute with HAMi
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!