Understanding the abilities of AI systems: Memorization, generalization, and points in between
summary
Argues that LLM abilities are best diagnosed by comparing outputs to training data and shows concrete tests and numbers: generated text has 93% novel 10‑grams and 63% novel syntactic structures relative to GPT‑2’s training set, so sentence formation is genuinely creative; algorithmic tasks instead track corpus frequency, with counting errors and a clear ROT13 spike at shift 13 and accuracy from 2% to 83% across probability bins, and chain‑of‑thought only partially shifts models from memorization toward noisy reasoning.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:41Motivation for AI abilities
- 1:44Clever Hans effect in AI
- 3:56Generalization lens for AI
- 6:02Memorization vs generalization in LLMs
- 7:50Case study: GPT-2 linguistic structure
- 16:19LLMs vs training data comparison
- 18:28Case study: Algorithmic reasoning with swaps
- 23:20Frequency effects in word lists
- 27:35Frequency bias in cipher decoding
- 32:30Chain of thought limitations
- 37:52Summary and future directions
- 39:16Acknowledgments
- 39:41Q&A
related talks
17mPsychology of AI
01h 18mThe Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes
24mHarnessing the Power of LLMs in Advanced Analytics
58mMirror, mirror: LLMs and the illusion of humanity
46mEvaluating Patterns in Gen AI Outputs
25mWhy LLMs Still Fail: Reasoning, Robustness, and Safety
59mMirror, mirror: LLMs and the illusion of humanity
40mMeasuring the unmeasurable: How to evaluate generative AI-based systems?