PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated
Talkat 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21) · 2021-07-14..07-16·
summary
PET applies partially equivalent transformations plus automated corrections to expand the tensor-program search space and finds faster kernels by combining operator-, tensor-, and graph-level rewrites. It uses superposition-based mutant generation, exploits ML tensor multilinearity theorems to reduce equivalence checks to a few random tests, and auto-generates correction kernels that add <1% overhead. PET builds a search-based optimizer (optimizes a model in <30 minutes) and shows up to 2.5× speedups (1.2× on ResNet-18); code is public.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:43Tensor Program Transformations
- 0:57Current Systems Consider only Fully Equivalent Transformations
- 2:33Motivating Example
- 4:42PET Overview
- 5:18Key Challenges
- 5:51Mutant Generator
- 6:46Challenges: Examine Transformations
- 7:07A Strawman Approach
- 7:41Multi-Linear Tensor Program (MLTP)
- 10:53Mutant Corrector
- 12:11Program Optimizer
- 13:33More Evaluation in Paper
related talks
18mPetS: A Unified Framework for Parameter-Efficient Transformers Serving
15mEINNET: Optimizing Tensor Programs with Derivation-Based Transformations
11mQiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a...
12mPreview: ML Serving
24mVerified Tensor-Program Optimization Via High-Level Scheduling Rewrites
15mEnabling Tensor Language Model to Assist in Generating High-Performance Tensor...
21mKTransformers: Unleashing the Full Potential of CPU/GPU Hybrid Inference for M...
21mPecan: Cost-Efficient ML Data Preprocessing with Automatic Transformation...