SkyRay: Seamlessly Extending KubeRay to Multi-Cluster Multi-Cloud Operation
Talkat Cloud Native + Kubernetes AI Day NA 2024 · on Tuesday, 2024-11-12· at 09:10· Salt Palace | Level 1 | Grand Ballroom A·
summary
Extends KubeRay to schedule Ray clusters, jobs and services across multiple Kubernetes clusters and clouds by interoperating with a policy-driven fleet manager. Uses KubeRay CRDs placed by a Nova fleet manager and leverages a Luna autoscaler for just-in-time node provisioning. Demonstrations include gang-scheduled training where two of three Ray jobs run on a static cluster and the third triggers autoscaling, LLM serving that falls back from GKE to EKS by priority, and clone-based cluster upgrades — all implemented and validated.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 2:29Sky computing and SkyRay
- 3:55Extending KubeRay with fleet
- 8:30Six SkyRay use cases
- 13:50Production vs development policies
- 15:58Multi-cloud priority and spot policies
- 18:50Zero-downtime Kubernetes upgrades
- 20:59Compound AI pipeline with SkyRay
- 23:04Conclusion and open-source
related talks
39mSailing Ray Workloads with KubeRay and Kueue in Kubernetes
19mScaling Frontier AI with Ray on Kubernetes
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
28mRay on Public-Cloud k8s - experiments, lessons & best practices
31mManaging Large-Scale Ray Deployments: Cloud, On-Prem, and Kubernetes
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
32mAn Overview of CloudKitchens's Ray-Powered ML Platform
17mRay History Server: Observability for Ephemeral K8s Clusters