Build Your Own Transformer
summary
90 minute hands-on workshop that implements self-attention from scratch in a shared Colab notebook using only NumPy. It walks a 3‑token example ("you are amazing"), uses 8‑dim random embeddings, initializes 8×8 Q/K/V weights, computes scaled dot‑product attention with masking and softmax, and outputs a 3×8 attention matrix. No large‑model training — weights stay random — but the Colab demo runs end‑to‑end and shows the exact math and code for each step.
AI-written summary · about
chapters (18)
- 0:00Workshop introduction
- 3:22Transformer architecture basics
- 13:39Setting up Colab
- 14:00Word embeddings from scratch
- 20:58Positional encoding
- 25:11Input matrix stacking
- 28:03Query key value matrices
- 39:20Attention head weights
- 42:24Q K V matrices
- 45:56Attention scores
- 50:12Softmax scaling
- 58:28Attention masks
- 1:04:27Softmax probabilities
- 1:08:53Attention output
- 1:13:29Multi-headed attention
- 1:19:29Encoder decoder layers
- 1:21:15Summary
- 1:22:43Q&A
related talks
28mEverything you need to know about Transformer Models
36mNeural Networks
26mA Shallow Introduction to Self-Attention
01h 25mPaper Discussion: Attention is all you need
01h 40mA Deep Dive into NLP with PyTorch
26mRethinking the Transformer: Toward Native Multimodal Architectures
38mUnderstanding and Applying Self-Attention for NLP
50mSequential Attention-Based Neural Machine Translation