Serving Qwen at Scale: Multi-Cluster AI Infrastructure on Karmada

on YouTube ↗04m

" summary

Covers how Qwen’s infrastructure teams run training and serving for large language models across multiple Kubernetes clusters and regions, unifying fragmented GPU resources and scheduling AI workloads at massive scale. Describes the platform architecture, reliability practices and lessons learned, and how cloud-native multi-cluster tooling—specifically Karmada—is used to make a fleet of clusters behave as a single AI platform.

AI-written summary · about

" related talks

• 27mGen-AI at Scale: Simplifying Orchestration of Healthcare Applications Across Multi-Cluster EnvironmeChristopher J Nuland, Selvi Kadirvel · Cloud Native AI Day Europe 2024
• 27mBalancing Cost and Efficiency: Day2 Optimization of Multi-Cluster AI InfrastructureKevin Wang · KubeCon + CloudNativeCon Europe 2025
• 25mHow We Power the Largest AI Deployments on the Planet: Running Vir...Lukas Gentele, Brandon Jacobs · Kubernetes AI + HPC Day North America 2023
• 36mMulti-Cloud Machine Learning Data and Workflow with KubernetesLei Xue, Fei Xue · KubeCon + CloudNativeCon China 2019
• 17mDevOps for Private AI: A Scalable Platform Using KubeflowMarco Cristofanilli · Incontro DevOps Italia 2025
• 36mMulti-Cloud Machine Learning Data and Workflow with KubernetesLei Xue, Fei Xue · KubeCon + CloudNativeCon + Open Source Summit China 2019
• 04mProject Lightning Talk: Karmada Updates and Use CasesHongcai Ren · KubeCon + CloudNativeCon Europe 2025
• 18mKeynote: Optimizing AI Workload Scheduling: Bilibili's Journey To an Efficient Cloud Native AI PlatformKevin Wang, Long Xu · KubeCon + CloudNativeCon China 2025