Under the Hood of LLMs — all those questions you never dared ask about them
summary
Describes why large language models are trained on enormous corpora (about 13 trillion tokens) and why they use subword tokenization rather than characters or whole words, using BPE with a ~256k vocabulary as a concrete example. Covers generation sampling methods (temperature, top-k and top-p) and explains persistent nondeterminism due to floating-point and parallel GPU execution, lists concrete failure modes (miscounting letters, hallucinations, fine-tuning data leakage), and presents a concrete experiment: fine-tuning GMA on dolphin audio.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 2:53LLM letter counting failures
- 10:19BPE token creation
- 16:36LLMs as parrots
- 19:45Foundation vs instruction-tuned
- 23:21Token selection & hyperparameters
- 27:59Determinism in LLMs
- 30:30LLM hallucinations
- 33:39Context impact
- 35:23LLM limitations
- 38:15Multilingual semantics
- 40:23Dolphin communication
- 41:27Q&A
related talks
46mThings you never dared to ask about LLMs
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
30mLearning to wield large language models
43mBeyond the Hype: A Realistic Look at Large Language Models
51mTowards a Pragmatic Understanding of Large Language Models
27mWhich Is To Be Master? Understanding LLM Tokenization
30mUnderstanding the end-to-end LLM training and inference pipeline
19mSystem for Scaling Large Language Models