Inside VLLM: Production Best Practices, Model Integration and Road Map
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
vLLM presents concrete production moves: a new Rust frontend, a KV connector, and staged reinforcement learning rollouts aimed at lower latency and easier state management. The roadmap claims immediate model onboarding with day-zero K3 support and walks through the integration mechanics for Kimi K3, showing a live demo of K3 serving successfully with the platform. Expect implementation details about deployment patterns, scaling knobs, and how the connectors plug into existing infra.
AI-written summary · about
related talks
39mAccelerating Open-Source RL and Agentic Inference with vLLM
35mThe State of vLLM
28mChildren's Guide To LLMs on Kubernetes
23mFast Inference, Furious Scaling: Leveraging VLLM With KServe
34mImprove AI Inference (serving models) With KServe and VLLM
20mInferencing LLMs in production with Kubernetes and KubeFlow
05mEvolving KServe: The Unified Model Inference Platform For Both...
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone