Keynote: Building AI in the Open: How the Cloud Native Community Shapes What Comes Next
summary
Kubernetes adds a production layer: Dynamic Resource Allocation GA is live and vendors must write external drivers, and Kubernetes AI conformance (launched at KubeCon NA) adds accelerator, DRA and networking requirements plus automated go-test certification. New projects build the runtime stack: LLMD inferenceuler orchestrates KV locality, disaggregated prefill/decode, autoscaling and multi-GPU parallelism; Leader-Worker Set operator groups pods as one compute unit (e.g., 16 GPUs) to preserve prefill/decode ratios during rolling updates. OpenTelemetry records evals and agent traces; Himei prototypes GPU partitioning and scheduler hints using DRA consumable capabilities.
AI-written summary · about
chapters (9)
- 0:00AI in Cloud Native projects
- 1:05Dynamic resource allocation for AI
- 3:33Kubernetes AI Conformance launch
- 6:26Creating CNCF conformance programs
- 8:57LLMD project for AI orchestration
- 14:51Leader Worker Set operator
- 21:00AI observability with OpenTelemetry
- 24:35GPU partitioning for small LMs
- 27:28AI adoption challenges panel
related talks
10mKeynote: Enabling AI with Cloud Native
09mKeynote: Kubernetes for GenAI: The New Operating System of the AI-First World
23mThe State of Cloud Native: The Shift Towards AI
11mThe Future of Cloud Native and AI
06mSponsored Keynote: Building Your AI Factory: Wh...
25mExtending Kubernetes for AI | Lessons Learned From Platform Engineering
17mKeynote: Open Source × AI: Shaping the Future of kt cloud
09mSponsored Keynote: Production AI Runs on Open Sou...