Tidal Auto-scaling for Training and Inference Based on Kubernetes + KEDA
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
Describes a Kubernetes+KEDA-based system that dynamically reallocates GPUs between vLLM inference and LoRA incremental training to match traffic patterns. During peaks KEDA scales out inference pods to meet latency SLOs while running training jobs save checkpoints and release GPUs; when load falls the training jobs automatically resume from those checkpoints.
AI-written summary · about
related talks
42mSmart scaling with Keda
23mEffortless Scalability: Orchestrating Large Language Model Inference with Kubernetes
45mYour K8s Infinity Gaunlet: The Marvel of KEDA and Cluster-API for Infinite Scale
35mOptimizing Metrics Collection & Serving When Autoscaling LLM Workloads
32mScaling Generative AI Inference with llm-d
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
37mKubernetes Event-driven Autoscaling with KEDA
34mAutoscaling Made Easy with Kubernetes Event-Driven Autoscaling (KEDA)