Running WebLLM in the Browser on RISC-V Toward Lightweight, Local AI Experiences
summary
Demonstrates a working WebLLM WebAssembly port on a full RISC‑V stack by enabling TensorFlow.js and ONNX Runtime Web backends and running inference in the browser on a DeepComputing Roma 2 mainboard (SiFive P550 series, up to 64 GB RAM). Shows Chromium (not full Chrome) gaining web‑serial support so microcontrollers can be programmed and graphics feel usable, but measures a performance ceiling due to WASM SIMD128 vs scalable vectors and missing upstream kernel support; effort ships free hardware to 100 projects to seed software.
AI-written summary · about
chapters (7)
- 0:00Introduction and context
- 1:27RISC-V hardware evolution
- 3:58Challenges for AI on RISC-V
- 7:57Hardware sponsorship initiative
- 10:27Q&A session
- 13:42Avoiding Chromebook pitfalls
- 14:46WebAssembly SIMD limitations
related talks
17mWebLLM: A high-performance in-browser LLM Inference engine
34mPrivacy First: Building LLM-Powered Web Apps with client side WASM
31mFrom ML to LLM on device AI in the browser
16mReact + WebLLM: Building Intelligent Applications with In-Browser Language Models
46mFrom ML to LLM: on-device AI in the browser
30mP2P Distributed LLM Inference Across Browsers With llama.cpp And WebRTC
20mFrom ML to LLM: On-Device AI in the Browser
53mWebAssembly