Open Source Meets Production MaaS: GMI Inference Engine and Karmada for Global AI…
summary
GMI Cloud runs a Kubernetes-native inference platform combining GMI Inference Engine with Karmada to govern 40+ regions and 5+ GPU vendors, claiming one global scheduler for all GPUs and a 90% control‑plane footprint reduction by grouping regions into macro regions. Design details: GPUs present as a uniform BRS resource, global control cluster performs filter+score across five dimensions, gateways route via the cloud backbone, and elastic page‑fault autoscaling provisions GPUs on demand.
AI-written summary · about
chapters (19)
- 0:00Global AI inference platform intro
- 1:06Four platform challenges
- 2:30Control and region clusters design
- 5:02Unifying GPU resources across clouds
- 7:32GPU worker connectivity tunnel
- 7:55Cluster and GPU scheduling split
- 9:44GPUs as global resource pool
- 10:29Scheduling page and compact operations
- 12:59Global routing with cloud backbone
- 16:16Elastic scaling control loop
- 18:11Platform workflow summary
- 19:17Key platform results
- 20:43Karmada for multicluster ops
- 21:24Open source contributions future
- 22:05Summary final message
- 22:26Q&A: Propagation policy CRDs
- 22:46Q&A: Failover resource interpretation
- 23:06Q&A: Resource distribution view
- 24:56Q&A wrap-up
related talks
20mOperating Frontier Intelligence at Scale
57mBuilding GPU-accelerated multi-agent apps
09mDemocratizing Access to GPUs for Faster MLAI Innovation with Generative AI
34mKServe in Production: Scaling Generative Inference the Cloud Native Way
34mScaling AI Like a Pro-PepsiCo’s LLM Deployment Strategy on K...
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...
30mGreen AI in Cloud Native Ecosystems: Strategies for Sustainability and Efficiency