Your GPU delivery is here! Now what? A 5 minute guide to make your CTO happy
summary
Company spends ~$8M on four DGX nodes (about 32 H100s) and sees roughly 40% idle on some teams while others hit node limits. Solution: run a shared cluster but expose per-team control planes via vCluster — a Kubernetes control plane running in a pod that maps to namespaces and can surface controllers like RunAI, Kubeflow or Volcano. Add vNode to isolate workloads at node level without VMs, claim stronger isolation than gVisor, and increase GPU utilization across tenants.
AI-written summary · about
related talks
43mThe GPU Utilization Problem: What’s Going Wrong and How to Solve It
24mCluster Management for Large Scale AI and GPUs: Challenges and Opportunities
28mShared Yet Isolated at Scale: Building Multi-Tenant Inference Platform on…
38mImproving GPU Utilization using Kubernetes
31mAI, CERN, and the Quest for GPU Custody: How CERN Leverages DRA for Efficient GPU Sharing
21mExperience Operating Large GPU Clusters at Organizational Scale
22mKeep HPC Running - an SRE's Guide to Supporting GPUs on Kubernetes
28mGPU Reservations: Maximizing Utilization and Fairness Across Teams