Lightning Talk: A Stable Limited LibTorch ABI? How?! (and Why?)
summary
Proposal to add a limited stable libtorch ABI that lets C++/CUDA extensions survive PyTorch upgrades by building against a promised surface rather than rebuilding every release. Plan uses a C shim with opaque AOTI tensor handle pointers, a C++ convenience layer called torch stable tensor, and stable IValue marshaling so diverse schemas share one dispatcher. About 2,000 ATen ops map into a single torch_call_dispatcher, FlashAttention 3 is already migrated, and a public wheel built on a nightly with CUDA 12.9 works with torch >= 2.9.
AI-written summary · about
chapters (14)
- 0:00Stable LibTorch ABI intro
- 0:21Unstable LibTorch API problems
- 1:23Limited stable ABI concept
- 2:25C shim for stable APIs
- 3:51UX challenges of C shim
- 4:11C++ stable wrappers
- 5:13Stable torch macros
- 6:38AOTI ops dispatcher
- 7:41Stable device utilities
- 8:03Header-only stable APIs
- 9:05New LibTorch ABI architecture
- 9:47Flash Attention 3 migration
- 10:30Status and future steps
- 11:32Q&A
related talks
18mtorch.accelerator: A Unified, Device-Agnostic Runtime API for Stream-Based Accelerators
36mDL Compiler Panel Discussion
10mLightning Talk: Faster Than SOTA Kernels in Torch.compile With Subgrap...
11mLightning Talk: Torch-Spyre: Compiling To a Multi-core Dataflow Accelerator...
01h 29mC++ API & ABI versioning
56mLightning Updates
07mLightning Talk: Towards supporting backward compatibility
16mLightning Talk: Triton Compiler