GPU sharing done right: Secrets, security, and scaling with Vault and vCluster
summary
Shows a concrete stack: vCluster for per-team virtual clusters, HashiCorp Vault namespaces with listener route per-namespace, and GPU sharing via MPS or MIG (hardware MIG 7 partitions). Demonstrates Kai scheduler routing GPU jobs, turns ~49% pooled GPU utilization into high‑90s efficiency, and issues dynamic, auditable AWS/GPU credentials (example lease: 4 hours). Includes live integration details: syncer behavior, Vault policy examples, and how auditing and Kubernetes-version migration work end to end.
AI-written summary · about
chapters (19)
- 0:00Introduction and setup
- 1:28GPUs and Kubernetes basics
- 4:13GPU sharing with MPS
- 5:36Hardware-level isolation with MIG
- 8:04Dedicated GPU clusters per team
- 9:29Shared clusters with Kai scheduler
- 11:14vCluster for multi-tenancy
- 13:00Isolation vs efficiency trade-offs
- 14:23Combining vCluster and Kai scheduler
- 16:09Vault for secrets management
- 17:31Vault namespaces for multi-tenancy
- 19:15Time-sliced GPU access with Vault
- 20:58Defense in depth and isolation
- 22:45Virtual clusters and blast radius
- 23:49Auditing in virtual clusters
- 24:50Scaling beyond vCluster
- 25:52Kubernetes version upgrades
- 26:35GPU fairness challenges
- 27:17Eliminate forever credentials
related talks
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
09mLightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads
34mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
28mShared Yet Isolated at Scale: Building Multi-Tenant Inference Platform on…
38mImproving GPU Utilization using Kubernetes
28mGPU Reservations: Maximizing Utilization and Fairness Across Teams
22mBeyond Single-Cluster Limits: Scaling GPU Workloads Across Kubernetes With…