Transparent Open Source AI Video Analytics with Panfrost
summary
Pure open-source stack running on Rock Pi 4 uses the Panfrost driver for Mali GPUs, TensorFlow Lite with the TensorFlow Lite GLES delegate and an ONNX-based gstneuralnet GStreamer element to offload inference to the GPU. An Optimised YOLO model runs on-device at ~83% accuracy, occasionally confusing prams for motorcycles; explainability work uses saliency maps, perturbation tests and layer-wise relevance propagation to trace why specific misclassifications occur. Cooling and quantisation/pruning are applied to meet edge constraints.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:20Welcome
- 1:53Object Detection
- 7:23AI on an Edge Device
- 11:31TensorFlow Lite
- 16:26Explainability
- 23:55Why
- 24:06Reason Codes
- 28:30Lack of Open Source
- 31:33Visualizing Data
- 33:35Saliency Maps
- 35:11Story
- 37:11Explainable Models
- 37:54Layerwise Relevance Propagation
- 38:40Recommendations
- 39:33Conclusion
related talks
20mComputer Vision at the Edge with OpenVINO
18mDeploying ML solutions with low latency in Python
48mLeveraging the Power of C++ for Efficient Machine Learning on Embedded Devices
09mLightning Talk: Streaming and Processing Edge Vision Data in Real Time
57mBuilding GPU-accelerated multi-agent apps
45mExtracting real-time data at the edge with SageMaker & NVIDIA GPUs
07mDemo: Neural Network Acceleration on Metis, Powered by RISC-V
24mMingfei Huang: Accelerating AI from Cloud to Edge