Compiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
summary
SubCuber compiles Strassen-like algorithms into CUDA by building the algorithm dependency graph, choosing tile-wise schedules, and applying recursion-aware fusion to emit two novel kernels (K3 and K4 kernels) that overlap input-sum loads with matmul work and cut memory traffic. It uses Cutlass primitives for Ampere/Hopper GPUs and evaluates on A100 and H200, achieving up to 12% faster (one level) and 22% faster (two levels) than Cutlass/cuBLAS, with 2–5× reduction in input-sum time and up to 16% end-to-end LLM inference speedups.
AI-written summary · about
chapters (15)
- 0:00Fast matrix multiplication intro
- 0:21Matrix basics and Strassen
- 1:45Subcubic algorithms on GPU
- 4:54Compiler for Strassen-like
- 5:58SubCuBE design and kernels
- 7:26Optimizing input sums with K3
- 8:49K4 kernel and cache-aware
- 9:52Kernel schedules and transforms
- 11:15Hardware and kernel details
- 12:15FP32 performance results
- 13:19FP16 with Tensor Cores
- 14:21LLM inference speedup
- 15:25Summary and future
- 15:46Q&A: Numerical stability
- 17:10Q&A: StreamK vs launch latency
related talks
01h 00mBringing Clang and C++ to GPUs: An Open-Source, CUDA-Compatible GPU C++ Compiler
10mMaximizing Kernel Development Productivity Under Performance Constraints
25mLeveraging AI Ecosystem for Portable and Sustainable GPU Kernels in HPC
29mBeyond CUDA: GPU Accelerated Computing on Cross-Vendor Graphics Cards with Vulk...
01h 26mGPU Algorithm Authoring with CUDA Tile
40mHidden GEMMs: How Optimized Math Libraries Work
01h 00mThe Shape of Things to Come
26mModeling and Analyzing Evaluation Cost of CUDA Kernels