Beyond Model Sharding: Atomic Scheduling and Disaggregated LLM Serving With ...
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
Describes LeaderWorkerSet (LWS), a Kubernetes-native API for treating a correlated group of pods across nodes/GPUs as a single workload unit for multi-host LLM inference. Explains gang scheduling integration using PodGroup with Volcano, scheduler-plugins, and YuniKorn to co-schedule leader and worker pods, and presents DisaggregatedSet for coordinating disaggregated prefill and decode resources to avoid rollout, scaling, and placement mismatches that harm latency and GPU utilization.
AI-written summary · about
related talks
23mNew Pattern for Sailing Multi-host LLM Inference
06mProject Lightning Talk: Sailing Multi-Host Inference with LWS
21mBeyond the Theory: What Actually Breaks When You Scale Your Disaggregat...
29mOptimizing Training Performance for Large Language Model(LLM) in Kubernetes
05mProject Lightning Talk: Extend Large Language Model Training Beyond Single Kubernetes Cluster
27mServing PyTorch LLMs at Scale: Disaggregated Inference With Kubernetes and Llm-d
31mSIG Scheduling Update: Transition From Pod To Workload Scheduling
30mCloud Native AI: Harness the Power of Advanced Scheduling for High-Performance AI/ML Training