Managing Large-Scale Ray Deployments: Cloud, On-Prem, and Kubernetes
summary
Anyscale describes a platform that runs over 10,000 Ray clusters and routinely manages millions of cores and GPUs and demonstrates a single Ray cluster scaled to 800,000 CPU cores. Deployments use three concrete patterns: direct VM APIs with an instance manager and on-node daemon, on‑prem machine pools for bare‑metal lifecycle and priority preemption, and Kubernetes via the Anyscale Operator that installs with helm and talks to a regional control plane. Features include cgroup protection, GPU health probes, overlay networking for multi‑region bursting, and auto select workers that pick accelerator types.
AI-written summary · about
chapters (7)
- 0:00Introduction to Anyscale
- 0:55Cloud VM infrastructure
- 5:55Multi-cloud bursting
- 9:59On-premise machine pools
- 12:00Anyscale Kubernetes operator
- 16:17Operator deployment demo
- 23:02Conclusion and Q&A
related talks
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
17mPlatform Orchestration: Anyscale + Ray on Kubernetes
19mScaling Frontier AI with Ray on Kubernetes
16mFrom Self-Managed Ray to Anyscale Services: Migrating ML Serving
20mThe Quick Journey to Using Ray: How We Implement Ray and Anyscale to Speed up our ML Processes
13mRay and Anyscale: An Optimization Journey
28mRay on Public-Cloud k8s - experiments, lessons & best practices
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes