Extending Kubernetes for AI | Lessons Learned From Platform Engineering
Talkat KubeCon + CloudNativeCon Europe 2025 · on Friday, 2025-04-04· at 11:45· Level 1 | Hall Entrance S10 | Room A·
summary
Teams describe extending Kubernetes with GKE, Cloud Run, and open source like Daxter for data workflows, plus custom controls for pod-level resources and in-place pod vertical scaling to resize running jobs without restarts. Use cases include Weaviate-style vector DBs with GPU-backed embedding services and a bursty transformation agent using Modal. Operational realities: fixed GPU pools (cloud capacity is not infinite), autoscaler lead times measured in months, 4 million CPU cores on Kubernetes today aiming for 8–10 million in two years, and heavy emphasis on checkpointing and preemption.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 5:41Customizing Kubernetes for AI
- 9:13Kubernetes resource allocation
- 12:47AI workloads on Kubernetes
- 17:59Maximizing GPU utilization
- 20:06Fixed GPU pools at scale
- 21:33Prioritizing AI workloads
- 22:35Compute choices for AI
- 24:00Uber’s Kubernetes with TPUs
- 25:05Q&A
related talks
27mHarnessing Kubernetes for Scalable AI/ML Workloads
43mKeynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
16mAI + Kubernetes: What Beginners...
40mPanel: Kubernetes and GPU Trends for AI in Financial Services
17mDevOps for Private AI: A Scalable Platform Using Kubeflow
06mKubernetes Love Machine Learning, Even on Private Cloud
38mKubernetes for AI Workloads – What Works and What… Doesn’t
40mBuilding Custom GPU Clusters at Scale: Using Kubespray To Create High-Performance AI Infrastructure