Evolving and Scaling Ray for the Next Generation of AI Workloads
summary
Ray Core now pushes control‑plane and driver limits by offloading high‑volume events, cutting RPCs and lock contention to deliver 1.6x task throughput and schedule 40k actors on 10k nodes, with 300x faster PG creation in one benchmark. Resource isolation via cgroups yields zero node failures from resource contention. GCS gains embedded RocksDB and active/passive failover for faster recovery, a beta history server for postmortems, topology‑aware placement groups for NVLink/TPU slices, RDT for multi‑hundred‑B weight syncs, and experimental gVisor sandboxes for RL.
AI-written summary · about
chapters (22)
- 0:00Introduction and overview
- 1:11Ray core's role and team intro
- 2:13Evolving AI workload trends
- 3:42Common workload scaling trends
- 5:07Scalability improvements overview
- 6:12Workload-based scalability focus
- 7:37Scheduling layer optimizations
- 9:43Placement group performance gains
- 10:26Actor scheduling challenges
- 11:30Predictability on unreliable hardware
- 12:14Resource isolation hardening
- 12:57GCS fault tolerance upgrades
- 14:23Failure transparency and debugging
- 15:26Hardware heterogeneity focus
- 16:30Rack-aware scheduling for GPUs
- 17:33Topology strategy API details
- 18:38TPU slice integration
- 19:23RDT transport improvements
- 20:47Sandboxing for agentic RL
- 21:49GVisor integration benefits
- 23:36Key takeaways and future plans
- 25:40Q&A and closing remarks
related talks
37mEvolving Ray Core: Scalability, Reliability, and Compiled Graphs
23mImproving Ray for Large-scale Applications
28mScaling Multimodal Data and Reinforcement Learning with Ray
28mA Deep Dive into Ray’s Scheduling Policy
26mRay: A General Purpose Serverless Substrate?
15mRay: A Distributed Compute Engine for AI
14mKeynote: The Future of Ray
31mRay at Scale: Apple's Approach to Elastic GPU Management