CUTLASS: A CUDA C++ Template Library for Accelerating Deep Learning...
summary
CUTLASS is an open source header-only CUDA C++ template library that implements high-performance GEMM and convolution kernels since 2017, claims parity with closed-source libraries, and reports over 10M+ downloads. It supports architectures through Hopper 4th-gen Tensor Cores, exposes asynchronous TMA multicast copies, producer/consumer and ping-pong warp-specialized kernels, a compile-time layout algebra called cute, a Python emitter/profiler for autotuning, and provides static asserts and a much reduced API surface for building custom kernels.
AI-written summary · about
chapters (12)
- 0:00Start
- 4:09Tensor Cores Operation: Fundamental Shape
- 5:40Asynchronous data movement between global memory and shared memory
- 8:54Cute Layouts and Layout Algebra
- 13:33Layout Transcription Examples
- 13:57Layout Mappings
- 15:17Key Design Ingredients of Cute
- 18:52CUTLASS 3 Conceptual GEMM Hierarchy
- 21:26CUTLASS 3.x API Entry Points
- 23:35CUTLASS 3: Collective Builders
- 29:45CUTLASS Tensor Core Performance
- 30:38CUTLASS Python Interface
related talks
09mLightning Talk: Harnessing NVIDIA Tensor Cores: An Exploration of CUTLASS & OpenAI..
28mGenerating State-of-the-Art GEMMs for Heterogeneous Hardware with...
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
08mLightning Talk: Achieving SOTA GEMM Performance: A CuTeDSL Backend for PyTorch Induc...
39mThe CUDA C++ Developer Toolbox
57mDeep Learning with C++
25mLeveraging AI Ecosystem for Portable and Sustainable GPU Kernels in HPC
59mUsing Template Magic to Automatically Generate Hybrid CPU/GPU-Code