Unlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
summary
HAMi presents a Kubernetes-native agent that manages heterogeneous AI devices (NVIDIA, Intel, Huawei Ascend) with GPU sharing, device-level observability, and scheduler hooks. It enables multiple containers on a single GPU, enforces QOS by suspending low‑priority workloads and reserving memory for high‑priority jobs, and supports flexible policies like NUMA affinity and binpack/spread. Integrations include Volcano and scheduler-plugin, and production adopters report meaningful increases in device utilization and pod density.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:04HAMi and AI infrastructure
- 3:21HAMi for AI device management
- 6:35GPU sharing in HAMi
- 8:13Resource allocation policies
- 10:51HAMi features and hardware
- 13:08Memory and scheduling
- 16:52Observability in HAMi
- 18:02LLM inference efficiency
- 19:50Kubernetes with HAMi
- 21:41Deployment and adoption
- 24:46HAMi community and future
- 26:53Q&A
related talks
05mHami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
28mSimplifying AI for Edge Compute with HAMi
21mUnlocking How To Efficiently, Flexibly, Manage and Schedule Seven AI Chips in Kubernetes
53mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
24mEmpower Heterogeneous Edge AI Acceleration with K8s