Balancing Cost and Efficiency: Day2 Optimization of Multi-Cluster AI Infrastructure
Talkat KubeCon + CloudNativeCon Europe 2025 · on Thursday, 2025-04-03· at 16:00· Level 1 | Hall Entrance S10 | Room B·
summary
Describes running Karmada plus Volcano to optimize multi‑cluster AI scheduling, comparing a lightweight resource model (graded node status) with a slower but topology‑aware Karmada estimator that queries member clusters for accurate GPU and network placement; introduces a preferred noExecute taint to avoid mass evictions and an eviction queue that pre‑schedules replicas before evicting; shows a first‑cut Volcano global fair‑sharing mapping using scheduling suspension to align federated and in‑cluster queues.
AI-written summary · about
chapters (12)
- 0:00Multi-cluster AI infrastructure
- 1:44Multicluster architectures needed
- 3:36Two-level system design
- 6:07Multicluster scheduling tradeoffs
- 9:43AI workload scheduling challenges
- 10:06Cluster failover mechanisms
- 15:59Improving cluster status detection
- 17:49New API for eviction
- 18:09Queuing workloads in multicluster
- 21:02Volcano Global queue management
- 23:57Future directions and feedback
- 24:22Q&A
related talks
27mEfficient Multi-Cluster GPU Workload Management with Karmada and Volcano
05mSponsored Keynote: Towards Clouds of AI Clusters
43mRun Your AI Workloads and Microservices on Kubernetes More Easily and Efficiently...
30mVolcano: A Unified Scheduling Platform for Cloud Native AI
08mNext-Gen AI Orchestration With Volcano On Kubernetes
04mServing Qwen at Scale: Multi-Cluster AI Infrastructure on Karmada
05mMulti-Cluster Orchestration System: Karmada Updates A...
34mMulti-cluster Orchestration System: Karmada Updates and Use Cases