Enabling Tensor Language Model to Assist in Generating High-Performance Tensor...
Talkat 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24) · 2024-07-10..07-12·
summary
A compiler-side framework converts tensor-program search into a generation task and implements a tensor language model that samples decision paths rather than full code. It trains a ~100M-parameter GPT-2–style model on large random samples of a designed tensor language, then uses iterative supervised fine-tuning with real latency measurements (200k GPU / 300k CPU measurements to converge) to build a generator. Results: matches fully tuned Ansor/MetaSchedule while compiling 61× faster, and yields 2.25× better kernels than Roller at equal compile time.
AI-written summary · about
chapters (8)
- 0:00Introduction and challenges
- 1:39Limitations of current methods
- 3:31Tensor language model overview
- 5:39Defining the search space
- 7:37Data sampling and pre-training
- 8:55Generating tensor programs
- 11:30Iterative optimization
- 12:40Evaluation and results
related talks
19mSystem for Scaling Large Language Models
19mGuided Tensor Lifting
01h 00mInside GPT – Large Language Models Demystified
11mQiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a...
28mFine-Tuning Small Language Models with Python
24mEfficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
44mHow to build your own Tiny Language Model from scratch
43mState of GPT