Debugging Machine Learning on the Edge with MLExray
Talkat Kubernetes AI Day Europe 2022 · on Monday, 2022-05-16· at 13:35· Pavilion 3, Room D | Level 2 | Central Forum·
summary
An end-to-end debugging platform that instruments deployed edge models via a small API and writes per-inference logs of per-layer latency and tensors, inputs/outputs, memory and device sensors, then diffs those logs against a cloud reference to produce a debug report. It surfaces concrete faults — channel swaps, resized/rotated inputs, high quantization MSE spikes, or a single convolution that explodes latency on some hardware — but requires code changes and log parsing; Pixie integration with eBPF auto-telemetry can remove that friction.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 0:15Michelles background
- 0:41Why debugging ML on the edge
- 1:58Traditional model
- 3:04Benefits
- 4:35Deployment
- 5:19What can go wrong
- 6:14MLExray
- 6:43Python API
- 8:19Reference Pipelines
- 11:05Issues
- 11:30Preprocessing errors
- 12:43MLExray assertions
- 13:52Quantization
- 15:05Reference Pipeline
- 16:05Hardware Requirements
- 17:07MLExray Demo
- 19:58Limitations
- 21:02Pixi
- 22:50Resources
- 23:17Questions
related talks
23mMLExray: Observability for Machine Learning on the Edge
10mML-based Log Analysis for Faster Debugging
28mGain better insights while training ML models with SageMaker Debugger
31mRunning Machine Learning Analytics On Traces
30mDebugging at Scale Using Elastic and Machine Learning
30mTrain ML models faster with better insights using Amazon SageMaker Debugger
26mML Monitoring & Observability
10mLightning Talk: Debugging Edge AI on Zephyr and Lessons Learned