Bringing AI to the Heterogeneous Edge with WebAssembly & ONNX
Talkat WasmCon 2023 · on Wednesday, 2023-09-06· at 15:45· Grand Ballroom G, Hyatt Regency Bellevue, 900 Bellevue Way NE·
summary
Demonstrates running ONNX models as WASM modules: converts a SqueezeNet model to ONNX, updates onnxruntime-rs and adds an ONNX backend to waCNN, then compiles a component for wasm-time/runwasi and runs local inference. The prototype loads a ~45 MB model into Wasm, preprocesses a 224×224 image, runs FP32 inference and softmax, and returns a 99% class score for a dog — code and PRs for runtime and wasm nn are upstreaming.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 4:53Edge AI challenges
- 8:02WebAssembly and ONNX solutions
- 11:34Live demo: Wasm & ONNX inferencing
- 19:58Future of WasmNN & model optimization
- 27:45Q&A
related talks
33mMachine Learning in the Browser: Fast Iteration with ONNX & WebAssembly
36mApplied AI on the edge
33mWebAssembly based AI as a Service with Kubernetes
22mMachine Learning with Wasm (wasi-nn)
32mWebAssembly Based AI as a Service on the Edge with Kubernetes
10mDeploying machine learning in wasm to CPU, GPU and NPU with the new wa...
27mAI inference on the Edge cloud using WebAssembly
25mLeveraging Wasm for Portable AI Inference Across GPUs, CPUs, OS & Cloud-Nativ...