VeRL: Extreme Optimization Practices for Ultra-Large-Scale MoE Models
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
VeRL presents engineering advances for ultra-large Mixture-of-Experts models using PyTorch FSDP, adding expert parallelism and sequence parallelism partitioning. It also covers system-level optimizations for long-sequence training—asynchronous RL, speculative inference acceleration, and extreme low-precision training—and techniques that achieve full alignment between training and inference to prevent training collapse.
AI-written summary · about
related talks
46mSpeeding Up LLM Training by Over 45%
24mverl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
25mOptimizing Reinforcement Learning at Trillion-Parameter Scale
25mVerl: A Flexible and Efficient RL Framework for LLMs
10mvLLM & Deepspeed Updates
12mSmartMoE: Efficiently Training Sparsely-Activated Models through Combiningand...
46mFlexOlmo: Open Language Models for Flexible Data Use
22mOptimizing Large MoE Inference on NVIDIA Blackwell: NVFP4, ADP, and DualPipe Strat...