How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS
summary
A full GPT-2 small inference is implemented in 600 lines of Vanilla JS that loads about 1.5 GB of model weights into the browser IndexedDB and runs locally, letting developers step through tokenization, attention matrices and MLPs in the devtools. The model uses 768‑dim embeddings with a 1024‑token context and 12 transformer blocks, maps predicted 768‑vector outputs to 50,257 token logits, and reproduces Hugging Face GPT‑2 next‑token results while exposing sampling options like greedy, top‑k and top‑p.
AI-written summary · about
chapters (18)
- 0:00Introduction to the Talk and Conference
- 1:24Mission: Breaking Clark's Third Law (Demystifying LLMs)
- 2:45Overview of the "Spreadsheets are All You Need" Approach
- 4:14Today's Approach: Vanilla JavaScript Implementation
- 4:47Background Needed for the Workshop
- 5:22Key Resources for the Workshop
- 5:50Simplified GPT-2 Diagram as a Roadmap
- 7:07Quick Tour of the JavaScript Implementation of GPT-2
- 12:20Understanding Large Language Models (LLMs)
- 17:47Tokenization: Splitting Text into Subword Units
- 33:12Embeddings: Token and Position Embeddings
- 56:51Attention Mechanism
- 1:02:57Multi-Layer Perceptron (MLP) and Backpropagation
- 1:16:07Iteration: Refining Predictions Across Blocks
- 1:17:49Language Head: Turning Embeddings Back into Tokens
- 1:23:18Chat GPT vs. GPT-2: Key Innovations
- 1:31:37Summary and Conclusion
- 1:35:13Q&A Session
related talks
41mInside GPT – The Maths Behind the Magic
01h 00mInside GPT – Large Language Models Demystified
47mInside GPT – Large Language Models Demystified
01h 01mInside GPT – Large Language Models Demystified
31mFrom ML to LLM on device AI in the browser
46mFrom ML to LLM: on-device AI in the browser
44mHow to build your own Tiny Language Model from scratch
37mExpanding the Capabilities of LLMs Through Community Collaboration