Lightning Talk: Every Millisecond Counts: The Fine-tuning Journey of an Ultra-Eff...
summary
NXP’s flow shows an end-to-end path from PyTorch or MATLAB exports into ONNX, two open source converters (ONNX-to-TFLite, ONNX-to-Quant) that map ONNX directly to TFLite flatbuffer schema and to target quant formats to avoid inserted float quant/dequant penalties. A scalable Neutron NPU core runs CNNs, RNNs and small transformers on chips from $1 MCUs to Linux MPUs, enabling models up to two billion parameters, and reports MLPerf-Tiny gains of about 7–17x on M33 and up to 169x speedup for ResNet versus CPU.
AI-written summary · about
chapters (7)
- 0:00NXP PyTorch ecosystem intro
- 1:45Supported devices and use cases
- 3:10eIQ Neutron NPU overview
- 5:21Tooling and deployment
- 8:55ONNX conversion tools
- 12:05NPU acceleration gains
- 13:09Closing and contributions
related talks
16mLightning Talk: Efficient Inference at the Edge: Performance You Need at the Lowest...
37mDeep Learning Neural Network Acceleration at the Edge
23mThe Race to the Bottom: Low Latency in the age of the Transformer
11mLightning Talk: Accelerating Experimentation...
18mDeploying ML solutions with low latency in Python
17mLightning Thunder: Supercharged PyTorch for Modern Hardware
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
14mOptimize your TensorFlow Lite models