Things you never dared to ask about LLMs
summary
Explains why LLMs work on tokens not letters, citing tokenizers like BPE and sentencepiece tokenizer, vocabulary sizes (Gemini ~260,000 tokens, GPT‑4 ~128k) and a claimed training corpus of 13 trillion tokens (equivalent to ~450,000 human years reading). Covers sampling math and knobs—temperature, top‑k, top‑p—why low temperature still isn’t fully deterministic (GPU floating‑point ordering), quantization formats like fp32/bfloat16/int8, failure modes (confabulation/hallucinations, the reversal curse), large context windows (Gemini up to 1–2M tokens) and surprising B64 “translation” ability.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:47LLMs beyond chatbots
- 6:25Counting letters mystery
- 11:48Tokens vs letters/words
- 21:08Foundation vs instruction models
- 25:41Next token with hyperparameters
- 31:08Deterministic outputs impossible
- 33:42LLMs and confabulation
- 37:54Reversal curse explained
- 39:40Emergent capabilities
- 42:11Q&A
related talks
46mUnder the Hood of LLMs — all those questions you never dared ask about them
27mWhich Is To Be Master? Understanding LLM Tokenization
24mHarnessing the Power of LLMs in Advanced Analytics
37mExpanding the Capabilities of LLMs Through Community Collaboration
02h 52mLow Level Technicals of LLMs
01h 18mThe Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes
49mBen Hall & Barbara Fusinska
30mUnderstanding the end-to-end LLM training and inference pipeline