Scaling Frontier AI with Ray on Kubernetes
summary
KubeRay is now the production Kubernetes operator for Ray and shows concrete wins: a history server that persists Ray logs and replays dashboards, native token auth via Kubernetes RBAC, and in-place pod resizing integrated with Ray autoscaler. Deployments scale training to 1,024 GPUs and run near‑trillion parameter pretraining. Roadmap items include Kubernetes Workload API scheduling, node label admission hooks to drive Ray topology placement, and an event forwarder for end‑to‑end observability.
AI-written summary · about
chapters (11)
- 0:00Welcome and introduction
- 1:09What is Cubray
- 2:13Progress and milestones
- 4:43Frontier AI research examples
- 7:33Kubray community progress
- 8:19Key features overview
- 10:49History server and Ray token
- 13:19Token authentication with Kubernetes
- 14:24In-place pod resizing
- 15:51Feature roadmap
- 17:57Q&A and closing
related talks
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
39mSailing Ray Workloads with KubeRay and Kueue in Kubernetes
24mSkyRay: Seamlessly Extending KubeRay to Multi-Cluster Multi-Cloud Operation
28mRay on Public-Cloud k8s - experiments, lessons & best practices
28mScaling Multimodal Data and Reinforcement Learning with Ray
26mUsing Ray On Large-scale Applications at Ant Group
31mNext-Generation Data Science Workflows Using Ray
31mManaging Large-Scale Ray Deployments: Cloud, On-Prem, and Kubernetes