Deploying PyTorch Models To the Browser and Beyond With Transformers.js
summary
Transformers.js exports PyTorch models to ONNX using torch.export (Dynamo), applies quantization and graph rewrites with ONNX Runtime GenAI and Onnx‑Slim, and runs them via a new C++ WebGPU runtime or WebNN. Result: 3–4x speedups from a fused multi‑head‑attention op, 4.4M monthly npm downloads, ~200 tokens/s on an M4 Max for a 1.25B model, and a 20B GPTOSS demo at ~40 tokens/s from a ~10 GB 4‑bit artifact — browser demos load, run locally, and are responsive.
AI-written summary · about
chapters (9)
- 0:00Failed YouTube spam to web AI
- 3:31Run models in browser
- 6:24Export PyTorch to ONNX
- 8:50Optimize ONNX with Dynamo
- 11:17ONNX Runtime performance
- 15:48Transformers.js API
- 17:37WebGPU and WebNN
- 19:41Live browser demos
- 22:52Creative apps and future
related talks
25mTransformers.js: Building Next-Generation WebAI Applications
18mTransformers.js: State-of-the-art Machine Learning for the web
17mRunning Intelligence at the Edge: Web AI in Action
33mUsing Transformers.js to Bridge the Gap Between AI Research and
51mMaking neural networks run in browser with ONNX
41mEscaping the Cloud: High-Performance AI in your Browser
20mTensorFlow.js Bringing Machine Learning to the Web and Beyond
22mBoring but Effective: WebAI That Ships