How We Power the Largest AI Deployments on the Planet: Running Vir...
summary
CoreWeave runs thousands of Kubernetes control planes across bare metal—3,000+ customer clusters on ~5,400 nodes and tens of thousands of GPUs—by virtualizing control planes with vcluster Pro and syncing tenant workloads to large shared worker clusters. The stack uses KNative, Argo CD, custom CRDs and BlueField DPU network isolation, plus an accelerated object cache and tensor rizer that cut a ~20 GB model load from ~60s to ~12–15s, supports scale-to-zero and on-demand burst provisioning, and automates day‑2 ops and GPU health verification.
AI-written summary · about
chapters (7)
- 0:00Intro
- 1:37How We Serve IML
- 7:23Kubernetes
- 12:25VCluster
- 18:23Syncing
- 21:56VCluster Pro
- 23:25Whats Next
related talks
27mHarnessing Kubernetes for Scalable AI/ML Workloads
14mKeynote: Building the Infrastructure that Powers the Future of AI
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
09mKeynote: Kubernetes for GenAI: The New Operating System of the AI-First World
28mScaling AI Infrastructure
20mOperating Frontier Intelligence at Scale
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
10mKeynote: Enabling AI with Cloud Native