Accelerating Open-Source RL and Agentic Inference with vLLM
summary
vLLM is an open-source inference engine that uses paged KV caching and continuous batching to cut KV memory overhead and boost utilization for multi-turn rollouts, adds sleep mode to swap inference and trainer weights on the same GPUs, and supports peer-to-peer weight sync so a 1T-parameter model can roll updates at scale in ~20 seconds. Evidence: 60k GitHub stars, 100k daily installs, 1,700+ contributors, and anecdotal speedups from 3 hours to 40 minutes on RL pipelines.
AI-written summary · about
chapters (23)
- 0:00vLLM and speaker intro
- 1:04Why vLLM exists
- 2:31Continuous batching
- 4:38KV cache memory issues
- 5:01Paged attention
- 7:28Prefix caching benefits
- 8:12vLLM growth impact
- 8:53vLLM interfaces
- 10:40Supported models
- 12:05Hardware backends
- 13:30Parallelism strategies
- 14:33How vLLM supports reinforcement learning
- 16:17RL pipeline inefficiencies
- 19:25Resource sharing sleep mode
- 22:10Efficient RL weight updates
- 23:36Scaling weight sync
- 25:03In-flight RL updates
- 26:47Determinism in RL
- 29:16DeepSeek R1 with vLLM
- 30:01Agentic models via Responses API
- 31:25Anthropic Messages API
- 32:26Dota 2 vLLM agent
- 33:52Join vLLM community
related talks
35mThe State of vLLM
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
24mverl: An Open Source Large Scale LLM RL Framework for Agentic Tasks
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
25mVerl: A Flexible and Efficient RL Framework for LLMs
21mScaling LLM Inference with vLLM
13mHow Open-Source vLLM Topped the Artificial Analysis Leaderboard
33mBeyond VLLM: Distributed LLM Inferencing With Llm-d on Kubernetes