Keynote: Kubernetes for GenAI: The New Operating System of the AI-First World
summary
Claim: Kubernetes is the production substrate for GenAI, not the LLM. Evidence includes OpenAI scaling to 7,500 nodes (2021), Google showing 65,000 nodes on TPU/GPU (April), and Amazon EKS adding 100k nodes (July 16). Concrete tooling: DRA (dynamic resource allocation) in 1.33/1.34, Nvidia MIG/VGPU, Kai scheduler, KServe/CloudNative projects, LLMD/VLM/Dynamo, Ray from Anyscale, KAgent and Google ADK, plus Kubernetes-native vector DBs and RAG pipelines driving real deployments and GPU utilization improvements.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:31What is Kubernetes
- 2:51The GenAI Stack
- 3:55GPU Challenges
- 4:49Model Inference
- 5:34Data and MCP
- 6:24Multiagent Orchestration
- 7:26Summary
- 7:45Enterprise adoption
related talks
43mKeynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
10mKeynote: Enabling AI with Cloud Native
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?- 10mKeynote: Enabling AI with Cloud Native
14mKeynote: Building the Infrastructure that Powers the Future of AI
32mKeynote: Powering Generative AI with Kubernetes
23mThe State of Cloud Native: The Shift Towards AI
27mHarnessing Kubernetes for Scalable AI/ML Workloads