Flexible Deployment of PyTorch Models on MCU-Class...
summary
ExecuTorch compiles PyTorch for Cortex‑M, Cadence DSPs, Ethos‑U and eIQ Neutron using portable C++ kernels, core‑specific backends and NPU delegates; benchmarks show 30–33× vs portable kernels on Cortex‑M, >100× on Cadence DSP and roughly 400× when moving to Neutron in one MobileNetV2 test. Recipe composition can double‑quantize operators (extra quantize/dequantize) unless nodes are pre‑partitioned and each operator is quantized to its execution backend. A demonstrated audio model reaches 1.4 ms NPU‑only and is tunable to 2.4 ms at 22 mW on i.MX RT700.
AI-written summary · about
chapters (11)
- 0:00Flexible deployment on MCUs
- 1:24MCU architectures and NPUs
- 2:30ExecuTorch backends for AI
- 3:13Benchmarking backend performance
- 5:00Handling unsupported operators
- 6:26Combining multiple backends
- 8:10Recipe-based quantization challenges
- 8:51Backend-aware quantization strategy
- 9:563ms inference with power limits
- 11:42Neutron NPU scalability
- 12:45Q&A: ExecuTorch vs alternatives
related talks
13mAI inference for the smallest processors
33mExecutorch: Enabling and Accelerating PyTorch Models on Edge Devices
27mDeploying GenAI for Audio Generation on Mobile CPUs With ExecuTorch
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
35mClient Side Deep Learning Optimization with PyTorch
22mIntroducing ExecuTorch from PyTorch Edge: On-Device AI...
11mLightning Talk: Accelerating Experimentation...
24mRunning AI/Neural networks on microcontrollers made simple with the STM32Cube.AI