Keynote: Welcome + Opening Remarks
summary
3,200 attendees from 800 organizations gather, with CNCF now housing 230 projects and India contributing 2.25 million cloud native developers; NVIDIA pledges $4M in GPU credits and upstreams a DRA driver. A new LLMD sandbox project demo uses vLLM + KServe on OpenShift AI serving Llama 3.1B and adds an inference routing layer that prefers pods with KV cache. Results show KV cache hit rate rising from ~33% to ~90% and P95 latency dropping from ~40s to ~8s on identical hardware.
AI-written summary · about
chapters (16)
- 0:00Welcome to KubeCon Mumbai
- 1:27CNCF growth and impact
- 3:11India in cloud native
- 5:58AI meets cloud native
- 7:01AI inference workloads rise
- 9:09Scaling AI on Kubernetes
- 10:55LLMD demo setup
- 17:52LLMD vs naive load balancing
- 21:26Why LLMD matters
- 22:30Specialized open source AI
- 24:39Uber’s AI model training
- 26:47Kubernetes AI certified
- 28:33Contribute to CNCF
- 30:00New education initiatives
- 31:45Community programs and events
- 33:10Closing remarks
related talks
04mOpening Remarks
01mClosing Remarks
06mWelcome + Opening Remarks
16mKeynote: An Open-Source Stack for AI Compute: Kubernetes + Ray + PyTorch + vLLM
05mSponsored Keynote: Towards Clouds of AI Clusters
06mKeynote: The Cloud Native Ecosystem in China
07mWelcome + Opening Remarks
12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA