Inside GPT – Large Language Models Demystified
summary
Shows GPT-2 on a laptop using Hugging Face + PyTorch to step through tokenization, embedding lookups and the 1x5x768 input tensor; demonstrates the 50k‑token vocabulary and examples like "floor" at 7.64% and "Lee" at ~99.98% probability. Explains positional encodings, 12 layers × 12 heads attention, and softmax math, and demos Word2Vec dimensionality reduction and RAG/chat history limits (4k/16k tokens). Visualizes how 768‑dim embeddings and temperature & top‑p (and frequency/presence penalties) change outputs and generation tradeoffs.
AI-written summary · about
chapters (15)
- 0:00Intro
- 1:59GPT sequence prediction
- 3:48Prompt engineering
- 4:45Demo: ChatGPT2
- 10:45Processing text
- 19:27Demo: Word2Vec dimensionality reduction
- 21:07Transformer architecture
- 23:58Demo: GPT2 input embedding
- 27:00Self attention
- 31:08Demo: GPT2 multi-head attention
- 32:52Attention example
- 33:49Demo: GPT2 next token prediction
- 44:06Parameters
- 46:21Thanks for explanations & inspiration
- 46:57Outro
related talks
01h 00mInside GPT – Large Language Models Demystified
01h 01mInside GPT – Large Language Models Demystified
41mInside GPT – The Maths Behind the Magic
51mLarge Language Models for Real-World Applications - A Gentle Intro
02h 50mUnderstanding and Using AI Language Models
01h 00mUnleashGPT: Mastering the Implementation of Large Language Models
25mEmbeddings, Transformers, RLHF: Three key ideas to understand ChatGPT
28mGPT or how I learned to stop worrying and love the generative AI