How IBM Research Achieved vLLM Platform Portability with Triton Autotuning
summary
IBM Research replaces hand‑tuned CUDA kernels in vLLM with OpenAI Triton kernels plus an autotuning cache called Triton djaVu, which records best block‑size configs into JSON and ships them in containers. Results show some kernels get 100% speed‑ups versus prior builds, Triton‑only vLLM reaches within ~8% of state‑of‑the‑art for large batches, and cold‑start autotune time drops from hours/days to seconds yielding effectively zero autotuning overhead.
AI-written summary · about
chapters (13)
- 0:00Introduction to platform portability
- 1:28Background on vLLM and Triton
- 3:55What is OpenAI Triton
- 7:27Triton in vLLM limitations
- 11:42Problem with autotuning in production
- 12:03Introducing Triton DVu framework
- 14:10Ensuring safe autotuning decisions
- 15:57Expanding configuration spaces
- 16:19Cross-vendor performance portability
- 18:51End-to-end performance results
- 19:53Startup time comparison
- 21:00Live demonstration
- 24:12Conclusion and future work
related talks
09mLightning Talk: Vllm-triton-backend: How To Get State-of-the-art Performance on...
24mPortable High‑Performance LLM Serving: A Triton Backend for...
26mVLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
22mEnabling VLLM V1 on AMD GPUs With Triton
10mAuto-tuning vllm
34mBridging the Gap: Compiling and Optimizing Triton Kernels Onto RISC-V Targets Based on MLIR
20mUnlocking Performance: Harnessing LLMs To Streamline GPU Kernel Development in...
15mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone