IoT Compute Unleashed: Transparent Offload From ESP32 To Edge Clusters
summary
ESP32 microcontrollers remotely run full YOLOv11 inference by offloading to a GPU-backed Kubernetes node using vAccel and a Twirp transport, dropping per-frame latency from ~1.4s on an ESP32-P4 (6s on an S3) to under 200ms. The system combines Akri for device discovery and inventory, OpenID-based attestation for secure onboarding, and OTA updates packaged as OCI firmware images via a flash-job operator; demos and videos show the pipeline working end-to-end despite flaky Wi‑Fi.
AI-written summary · about
chapters (14)
- 0:00Cloud-native IoT research intro
- 1:04Secure IoT device management goals
- 3:52IoT, cloud-native and security defined
- 5:40Cloud-native IoT framework overview
- 7:28Secure device onboarding with Akri
- 9:55OTA updates with OCI images
- 11:21Firmware pipeline with GitHub Actions
- 13:05IoT demo applications
- 16:01ML inference challenges on IoT
- 17:49vAccel for hardware acceleration
- 19:13Offloading YOLO inference to GPU
- 20:34Live demo of compute offloading
- 23:33Project status and future
- 24:14Q&A
related talks
29mLife on the Edge
49mEnable IoT with Edge Computing and Machine Learning
32mExtending machine learning to industrial IoT applications at the edge with AWS
39mExploring the Trend Toward the Edge
15mBringing Cloud Agility to the IoT Edge
52mLiving on the Cloud's Edge
31mBring Cloud Connectivity and Intelligence to the IoT Edge
55mFog Computing and IoT Solution Architecture