PyTorch Edge: Vendor Integration Journey for Compilers and Backends
summary
Describes concrete integration points for PyTorch Edge: a quantization API with prepare/calibrate/convert and an annotate() quantizer that emits dquan/quan nodes so vendors can target int4/int8 weights or fp16 activations (example: fused quantized linear+ReLU kernel), and a delegation API that compiles subgraphs to an opaque delegated payload embedded in the .pte file, initialized at load and executed via a runtime handle. Uses the Edge dialect (~180 operators), supplies debug handles to map failures back to Python, and lists successful integrations with Apple, Arm, and Qualcomm.
AI-written summary · about
chapters (9)
- 0:00PyTorch Edge team vision
- 2:27On-device AI deployment challenges
- 6:23Partner integration vision
- 10:00Vendor integration benefits
- 11:00Quantization and delegation entry points
- 13:32Quantization API walkthrough
- 17:20Backend delegation framework
- 20:11Delegation contracts and debugging
- 24:03Partner integrations and next steps
related talks
33mExecutorch: Enabling and Accelerating PyTorch Models on Edge Devices
25mLeveraging PyTorch for Generative AI in Distributed Edge Clouds
08mUnlocking AI at Scale for PyTorch Developers
59mBuilding models for TPU devices
23mBringing ExecuTorch To the Next Frontiers of Edge AI
25mExecuTorch 1.0: General Availability Status for Mobile and Embedded...
13mRunning ExecuTorch Applications With Silicon Accelera...
37mDeep Learning Neural Network Acceleration at the Edge