Shiftry: RNN Inference in 2KB of RAM
summary
Shiftry compiles RNN inference into C that runs on an Arduino Uno with only 2 KB RAM by converting floating point to fixed point, using mixed bitwidth assignment to trade accuracy for flash size, and by static allocation simulation to eliminate runtime fragmentation. It fits FastGRNN models that SeeDot cannot, produces code with ~1% better accuracy, ~40% lower latency and ~25% smaller size than SeeDot, and compresses a larger RNNPool to ~32× lower RAM and >3× lower flash so it runs on an ARM Cortex‑M4 class device.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:30IoT
- 1:00ML on IoT
- 2:06RNN on IoT
- 2:39Challenges
- 4:04Input DSL
- 5:59Assign Scales
- 7:18Shiftry
- 9:02Managing RAM
- 11:18Dynamic Memory Allocation
- 12:11C Dot Benchmarks
- 14:38RNN Pool
- 15:35Conclusion
related talks
26mMemory-Efficient LLM Inference on Edge Devices With NNTrainer
15mQuantization at the Edge: Making a 4GB Model Run on 1GB RAM
17mCMSIS-NN
01h 11mMachine Learning Acceleration in an Embedded World
25mCMSIS-NN
15mDesign Compact Deep Learning Models: Small is the New Big
39mLeveraging the power of C++ for efficient machine learning on embedded devices
24mRunning AI/Neural networks on microcontrollers made simple with the STM32Cube.AI