End-to-End Optimizing Multi-Turn RL and High-Performance Inference in Agents with...
summary
SGLang presents concrete methods and numbers: an oversampling + partial rollouts (APRIL) pipeline that cuts multi‑turn RL training time by about 40%, token/sequence/geometry level importance sampling to mitigate training–inference mismatch (and recover FP8 training accuracy), deterministic kernels that remove nondeterminism but only halve mismatch failures, and a CUDA graph aware weight offloading system plus torch memory saver for full GPU utilization when collocating SGLang (Eston) and FSTP/FSDP or Megatron; supports Llama4, gpt-oss, Qwen3, DeepSeek-R1.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:09Eston for agentic RL
- 8:55Multi-turn RL debugging
- 13:06Partial rollouts for efficiency
- 15:51Training-inference mismatch
- 21:28Mismatch mitigation with importance sampling
- 25:00Non-determinism in inference and training
- 26:25System design for sampling and training
- 29:57Memory recap optimization
- 32:06Final takeaways
related talks
19mTraining Agentic Reasoners
20mSGLang: An Efficient Open-Source Framework for Large-Scale LLM Serving
36mSkyRL: Democratizing RL Training for Researchers
42mPost-Training Small Language Models: the adventures of a practitioner
06mBuilding Frontier AI Infra: SGLang and Miles
14mScalable Solutions for Running Large Language Models
24mLLM Inference Optimization From Simple To Sophisticated
43mState of GPT