SkyRL: A Modular RL Library for LLM Agents
summary
SkyRL is an open-source RL stack that trains multi-turn, tool-using LLM agents with a modular trainer/generator/controller architecture, standardizing on the OpenAI API for inference and supporting FSDP and Megatron training backends. It provides an agent layer with configurable dispatch strategies (async pools, 3-stage pipelines), reusable tool patterns (search, web fetch, bash), and LoRA + SkyLTX integrations. Teams validate it end-to-end on deep research and VB-bench style tasks and report an improvement from 28% to 33% evaluation accuracy at the same number of steps.
AI-written summary · about
chapters (14)
- 0:00SkyRL introduction
- 0:42RL workloads for LLMs
- 3:28Need for modular RL
- 5:37SkyRL architecture
- 8:44SkyRL training features
- 10:08Generator in agent loop
- 10:51Environment integration
- 13:59Training on real tasks
- 17:12SkyRL agent stack
- 19:19Connecting stacks
- 20:01Efficient task dispatch
- 22:09Deep research case study
- 24:58Validation and results
- 25:40SkyRL roadmap
related talks
36mSkyRL: Democratizing RL Training for Researchers
25mVerl: A Flexible and Efficient RL Framework for LLMs
14mClosing the Train-Inference Divide for Asynchronous RL at Scale
14mRLlib: Scalable RL for TensorFlow, PyTorch, and Beyond
19mTraining Agentic Reasoners
39mTraining Frontier Knowledge Work Agents: A 397B Open Recipe with SkyRL
24mverl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
44mWhy RL Environments Are All You Need for Building Agents