Running Google's Gemma LLMs in the browser with MediaPipe Web
summary
MediaPipe Web runs Gemma 3 and Gemma 3N fully in-browser using WebGL/WebGPU and WASM, streaming models piecewise so even a 27B / 27 GB Gemma model can load and run on a 2024 MacBook Pro at near–human reading speeds. The system uses streaming loading and per-layer embedding caching, mixes FP32/FP16 to handle Bfloat16-trained weights, and exposes a short JavaScript API plus multimodal input (text, image, single‑channel audio). Demo code and HuggingFace Spaces are available; models require HF auth and license acceptance.
AI-written summary · about
chapters (12)
- 0:00Introduction to Gemma
- 0:21MediaPipe web overview
- 1:19Gemma 3 model family
- 3:06Concurrent development
- 3:44Implementation challenges
- 4:49Performance results
- 5:30Streaming loading system
- 6:52Handling 16-bit floats
- 8:09API and code demo
- 8:52Instruction tuning
- 10:26Multimodal capabilities
- 11:54Demo apps and conclusion
related talks
08mDemo: Gemma on-device with MediaPipe and TensorFlow Lite
40mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
20mFrom ML to LLM: On-Device AI in the Browser
46mFrom ML to LLM: on-device AI in the browser
14mGemma on mobile and web. Best and worst practices
44mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model
16mOpen models in the Gemini era
12mWelcome to Gemma 3