Getting Started with AI and WebAssembly
Talkat WasmCon 2023 · on Wednesday, 2023-09-06· at 15:00· Grand Ballroom I, Hyatt Regency Bellevue, 900 Bellevue Way NE·
summary
Shows concrete tooling and limits: WASI‑nn hostcalls let WebAssembly modules call TensorFlow, PyTorch, TensorFlow Lite and ggml backends via Wasmtime/WasmEdge and a named model path that streams a 10–15 GB Llama 2 ggml file off disk into the accelerator, bypassing the 4 GB wasm linear memory. Demos compile Rust inference to ~2 MB wasm, run YOLOv5/MediaPipe with OpenCV and FFmpeg preprocessing, and run tiny LLMs on CPU, RISC‑V boards, browsers and GPUs, with near‑native preprocessing speed but GPU inference still done by the host.
AI-written summary · about
chapters (8)
- 0:00AI and WebAssembly intro
- 3:51Machine learning quick intro
- 7:23AI inference in WebAssembly
- 15:53Runtime vs module inference
- 21:50WebAssembly AI standards
- 25:18LLM inference demo
- 30:39Resources and community
- 32:50Q&A
related talks
01h 04mDevelop, Build & Deploy Lightweight AI Services with WebAssembly
27mWasm Is Becoming the Runtime for LLMs
25mLeveraging Wasm for Portable AI Inference Across GPUs, CPUs, OS & Cloud-Nativ...
40mWebAssembly for AI Infra : A Lightweight, Fast, and Secure Alternative Approach
41mCreating Versatile AI Agents Through Wasm+Rust
32mEmpower WebAssembly ML Over Diverse AI Hardware Accelerators Outside the Browser
27mAI inference on the Edge cloud using WebAssembly
31mRunning AI workloads with WASM - is it production ready? [Panel]