TinyML at the Edge: Deploying and Optimizing AI Workloads on Zephyr RTOS
Talkat Open Source Summit Japan + AI_dev: Open Source GenAI & ML Summit + Automotive Linux Summit 2025 · 2025-12-08..12-10·
summary
Practical notes for tonight: Zephyr RTOS runs TinyML on over 19,000+ microcontrollers and supports multiple runtimes — choose TensorFlow Lite Micro for TinyNNs, microTVM when you need compiler-driven assembly optimizations, or emlearn/MLAN for tiny tree-based models; Kenning’s AutoML pipeline generates and optimizes models for target MCUs. Zephyr’s Linkable Loadable Extensions (LLEXT) enable hot-swapping models without reflashing; recommended optimizations include quantization and operator fusion, and benchmarking on device vs Renode reveals runtime bottlenecks.
AI-written summary · about
chapters (11)
- 0:00Introduction and agenda
- 1:39Why Zephyr for edge AI
- 4:13Zephyr RTOS key features
- 6:38Inference engines and runtimes
- 11:18Runtime decision matrix
- 13:06Deploying TFLM models
- 16:07AutoML for model building
- 23:05Extensions and OTA updates
- 26:07Hot-swapping considerations
- 29:22Optimization and profiling
- 33:57Q&A and conclusion
related talks
01h 53mTinyML: Machine Learning’s Future is Tiny and Bright
40mHands-on TinyML Deployment on RISC-V: Building an Optimized Edge AI Pipeline
25mTinyML as-a-Service
27mWhat’s TinyML good for
18mManagement of IoT TinyML Devices
40mHands-on TinyML Deployment on RISC-V: Building an Optimized Edge AI Pipeline - Corrected Audio
07mLightning Talk: Zephyr Portability with an AI Application on Very Different MCUs
21mThings I learned running neural networks on microcontrollers