⚡ Before VLLM Starts: Preflight Checks for LWS for LLM Inference on K8S
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
Defines an LWS-based startup gate that runs init-container preflight checks and keeps them decoupled from vLLM, with a concrete KEP implementation LWS KEP #813 as the case study. Targets three production gaps: init-phase peer discovery failures, unbounded recreate loops, and unclear terminal failure boundaries, and demonstrates catching unstable throughput and p99 latency jitter before serving starts. Provides an explicit checklist and decision logic to map failures to retry/reschedule/stop, and reports successful production detection of those failure classes.
AI-written summary · about
related talks
33mBeyond VLLM: Distributed LLM Inferencing With Llm-d on Kubernetes
20mInferencing LLMs in production with Kubernetes and KubeFlow
33mllm-d: Kubernetes Native Distributed Inferencing
42mStop Treating LLMs Like REST APIs
30mSecuring LLMs in Kubernetes: Best Practices
40mScalable and Efficient LLM Serving With the VLLM Production Stack
23mNew Pattern for Sailing Multi-host LLM Inference
36mMore Than Model Sharding: LWS & Distributed Inference