Are You Really Out of GPUs? How to Better Understand Your GPU...
summary
Shows how to spot real GPU waste with concrete metrics and tools: scrape per‑GPU utilization from dcgm_exporter, query Prometheus to find GPUs with utilization == 0 but a running pod, and visualize allocations and pending requests in Grafana so you can see cases like “8 total GPUs, 2 pending pods, 2 GPUs allocated but idle.” Describes an open source CLI/environment tool GNF and practical fixes: delete idle GPU pods, convert notebooks to jobs, and add alerts and dashboards for actionable reclamation.
AI-written summary · about
chapters (8)
- 0:00GPU utilization challenges
- 3:55Need more GPUs?
- 6:48Dynamic GPU provisioning
- 10:43Measuring GPU use
- 14:37GPU metrics and dashboards
- 18:51Reduce idle GPUs
- 21:21Q&A on GPU scheduling
- 31:05Workload-specific provisioning
related talks
43mThe GPU Utilization Problem: What’s Going Wrong and How to Solve It
22mKeep HPC Running - an SRE's Guide to Supporting GPUs on Kubernetes
38mImproving GPU Utilization using Kubernetes
31mShared GPU Scheduling & Proactive Autoscaling: A Production Blueprint for 1000+…
20mHow are You Utilizing GPUs? Best Practices on Managing GPUs in Azure
07mYour GPU delivery is here! Now what? A 5 minute guide to make your CTO happy
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
21mExperience Operating Large GPU Clusters at Organizational Scale