Simplifying AI for Edge Compute with HAMi
summary
HAMi provides a hardware-agnostic scheduling plane that carves GPU memory and compute so multiple agents run concurrently on small devices and time-shares when demand exceeds RAM. Demonstrated on three-node A100 clusters with 10 GB MIG slices and a CUDA hook for time-slicing, it enforces per-pod memory to avoid OOM cascades. Compares Jetson 64 GB unified memory and H200’s 141 GB HBM to NPUs (Axelera/DeepX) with higher perf/W, supports ~10–11 devices, and runs as a CNCF incubating project.
AI-written summary · about
chapters (10)
- 0:00HAMi and edge AI compute
- 1:24Edge compute challenges
- 3:09HAMi’s core capabilities
- 4:53Memory and compute slicing
- 8:47Modern MPUs and unified memory
- 10:12HAMi with Kubernetes
- 13:23Live demo of HAMi scheduling
- 19:56GPU memory requirements library
- 21:20Case studies and adoption
- 23:08Q&A
related talks
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
05mHami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
02mSession: ISMM - ISMM: Keynote 2 - Dr. Euicheol Lim
14mArm: One Platform for Agentic AI, from Cloud to the Edge
21mA New Architecture for the Intelligent Edge
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
46mWhen AI Meets High Performance Computing