RLBoost: Harvesting Preemptible Cloud Resources for Cost-Efficient Reinforcement Learning
summary
RLBoost harvests preemptible GPUs (cloud spot instances and spare cluster capacity) to run rollouts while keeping training on reserved full‑mesh GPUs, claiming 1.51x–1.97x higher training throughput and 28%–49% cost savings. It implements adaptive rollout offload to shift work by availability, pull‑based weight transfer to provision new workers quickly, and token‑level response migration to resume interrupted generations with near‑zero wasted tokens; RLBoost is implemented and open‑sourced and shows stable gains across real spot traces.
AI-written summary · about
chapters (12)
- 0:00RLBoost introduction
- 1:43RL workflow bottlenecks
- 3:27Separating inference and training
- 4:55Elastic resource challenges
- 6:22Hybrid architecture design
- 7:23System design challenges
- 8:27Handling preemption
- 9:31Partial response offloading
- 10:57Runtime load balancing
- 11:39Evaluation with spot traces
- 13:23Cost efficiency comparison
- 14:25Conclusion
related talks
15mRollArt: Disaggregated Multi-Task Agentic RL Training at Scale
19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training
20mEfficient Reinforcement Learning
19mResource Management Innovation for AI Workloads in Cloud Computing
17mScaling Reinforcement Learning
36mAccelerating AI training fleets with sched_ext
19mRL Environments at Scale
22mAWARE: Automate Workload Autoscaling with Reinforcement Learning in Production...