Optimizing Long-Tail and MoE Challenges in Reinforcement Learning with SGLang
summary
SGLang presents concrete fixes: partial rollouts use oversampling so the first ~80% of fast responses (arriving in 50% of time) train immediately while slow 20% are postponed or cached, reducing straggler waits; importance sampling corrects training–inference log‑prob mismatch (works even for FP8) with under 100 lines of code; a cuda-graph‑aware wake implemented as Torch Memory Saver preserves virtual addresses so CUDA graphs survive context switches, avoiding costly rebuilds and enabling fast refit between serving and training at multi‑thousand‑GPU scale.
AI-written summary · about
chapters (6)
- 0:00Optimizing RL with SGLang
- 2:54RL system life cycle
- 10:04Partial rolling outs in RL
- 13:38Training inference mismatch
- 20:24CUDA graph wake-up
- 25:00Summary & Q&A
related talks
20mSGLang: An Efficient Open-Source Framework for Large-Scale LLM Serving
44mIntroduction to LLM serving with SGLang
16mRevisiting Pipeline Parallelism for LLM Serving
06mBuilding Frontier AI Infra: SGLang and Miles
11mData Efficient Methods for Reinforcement Learning
20mEfficient Reinforcement Learning
36mLanguage Model Post-Training in 2025: an Overview of Customization Options Today
08mHacks to Make LLM Training Faster