VLLM-Helion: SOTA LLM Performance by Advanced Autotuning and Fine-grained Dispatching
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
vLLM integrates PyTorch Helion to produce pre-tuned, platform-specific kernels via an offline autotuner that searches representative input shapes and a fine-grained runtime dispatcher that selects a config per kernel invocation. The open-source framework includes kernel authoring, autotuning, config management and runtime dispatch; benchmarks show 1.1x+ quant GEMM on H100, ~1.5x speedup on small aux kernels and up to ~10% end-to-end gains for some models. Autotuning can take hours–days, stop time adds seconds, eager launches cost tens of ms; Triton and experimental QDSL backends are used.
AI-written summary · about
chapters (13)
- 0:00VLLM-Helion project intro
- 1:03VLLM and Helion background
- 2:48What Helion offers
- 5:00Why integrate Helion into VLLM
- 7:51VLLM-Helion integration
- 9:14Helion kernel in VLLM
- 12:04Helion in VLLM challenges
- 16:02Performance solutions
- 18:33Quantized linear kernel benchmarks
- 20:19End-to-end gains with Helion
- 21:21Auxiliary kernel improvements
- 22:48Status and adoption paths
- 24:13Resources and involvement
related talks
10mAuto-tuning vllm
24mHelion 1.0: A High-Level DSL for Performance Portable Kernels
31mHow IBM Research Achieved vLLM Platform Portability with Triton Autotuning
28mHelion: A High-level DSL for Kernel Authoring
09mLightning Talk: Vllm-triton-backend: How To Get State-of-the-art Performance on...
20mUnlocking Performance: Harnessing LLMs To Streamline GPU Kernel Development in...
24mOptimizing CPU LLM Inference in PyTorch: Lessons From VLLM
24mPortable High‑Performance LLM Serving: A Triton Backend for...