On Memorization of Large Language Models in Logical Reasoning
summary
Paper measures memorization on a procedurally generated Knights-and-Knaves benchmark and defines a per-sample memorization score that multiplies accuracy with sensitivity under local perturbation. Off‑the‑shelf LLMs interpolate fine‑tuning data (near‑perfect training accuracy) yet fail small structural edits, while chain‑of‑thought fine‑tuning and more examples boost both memorization and cross‑difficulty generalization. Probing shows internal representations encode truth labels, wrong‑answer fine‑tuning still often improves performance, and memorization correlates positively with test accuracy rather than purely harming reasoning.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:26Generalization perspectives
- 5:05Generalization in the LLM regime
- 7:46Underparameterization challenges
- 11:29Memorization in LLMs
- 13:24Memorization in reasoning
- 17:23Measuring memorization
- 25:25Knights and knaves benchmark
- 28:53Evaluating memorization
- 30:44Fine-tuning experiments
- 35:05Probing representations
- 36:54Brute-forcing puzzles
- 38:58Memorization vs generalization
- 41:13Future of evaluation
- 42:46Q&A
related talks
27mExploring the Mechanisms of LLM Reasoning Capabilities
39mUnlocking reasoning in Large language models
14mImproving Large Language Model Performance
27mBuilding Mini Minds Our Adventures in Teaching Small LLMs to Reason
29mThe Evolution of LLMs: From Imitation to Reasoning
01h 05mAdaptive Generalization: The Role of Dynamic Evaluation in Low-Resource Settings
31mMaking the most of test-time compute in LLMs
58mIn-Context Principle Learning from Mistakes