Project Lightning Talk: Sailing Multi-Host Inference with LWS
Lightningat KubeCon + CloudNativeCon Europe 2025 · on Tuesday, 2025-04-01· at 16:21· Platinum Suite | Level 3·
summary
LWS delivers multi-host model-parallel inference by grouping one leader pod with its worker pods into a single superpod (example diagram shows four superpods with one leader and two workers each), supports a dual-template for CPU-only leaders vs GPU workers, and enforces all-or-nothing restart plus fine-grained rolling updates so model shards stay consistent. It adds topology-aware placement for cross-node latency, integrates with vLLM and Limas, and reached v0.6.0 after initial release in February 2024.
AI-written summary · about
related talks
23mNew Pattern for Sailing Multi-host LLM Inference
39mBeyond Model Sharding: Atomic Scheduling and Disaggregated LLM Serving With ...
10mLightning Talk: Accelerating LLM Training on Cerebras Wafer-Scale...
33mAccelerating the LLM Lifecycle on the Cloud
38mKubernetes for Multi-Host Training and Inference: Workload Aware Sc...
27mServing PyTorch LLMs at Scale: Disaggregated Inference With Kubernetes and Llm-d
13mBuilding Reliable Infrastructure for LLM Workloads on K8s
31mThe Evolution of Multi-GPU Inference in vLLM