AI inference for the smallest processors
summary
Porting PyTorch to run inference on bare‑metal microcontrollers is practical: a workflow lowers models with ExeTorch (Zechrch) ahead‑of‑time, a delegate partitioner assigns ops to host CPU, accelerator CPUs or custom hardware, and a single static binary contains custom 8‑bit and 32‑bit operators. Targets have only megabytes of RAM, no OS, and limited cores so aggressive memory placement and quantization cut model size ~4×; a cycle‑accurate pre‑silicon model gives exact stack and timing traces and the ports run successfully on prototypes with 4‑bit weights and 8‑bit activations.
AI-written summary · about
chapters (9)
- 0:00AI on microcontrollers
- 1:27PyTorch and ExecuTorch
- 2:54ExecuTorch on embedded devices
- 5:02Bare metal constraints
- 6:50Hardware accelerator optimizations
- 7:34Memory management
- 8:38Quantization for smaller models
- 10:04Pre-silicon with VeriLog
- 11:50Key takeaways
related talks
10mExecuTorch on Microcontrollers: Deploying PyTorch To...
14mFlexible Deployment of PyTorch Models on MCU-Class...
56mHow to: Bringing up ExecuTorch on a RISC-V Microcontroller
56mHow to: Bringing up ExecuTorch on a RISC-V Microcontroller | Corrected Audio
11mLightning Talk: Accelerating Experimentation...
20mArchitecting AI: Efficient Models, Everywhere
24mRunning AI/Neural networks on microcontrollers made simple with the STM32Cube.AI
25mMachine Learning on Microcontrollers; Tiny, Useful and Fun