Language Models as World Models?
Talkat Understanding Higher-Level Intelligence from AI, Psychology, and Neuroscience Perspectives · 2024-06-24..06-28·
summary
Evidence shows transformer LMs store rudimentary, linearly decodable representations of entities and dynamic state: linear probes recover properties from hidden activations on controlled beaker/alchemy and text-adventure benchmarks, and pretrained models already contain much of that signal. Constructed fact vectors can be injected to change generation and predict when a model will hallucinate. A concretely implemented intervention model, REMEDI representation editing, edits activations to correct some factual errors. Limitations remain: many facts resist linear readout and models still hallucinate or contradict inputs.
AI-written summary · about
chapters (7)
- 0:00Language models and meaning
- 3:59Defining world representations
- 9:35Probing internal states
- 15:14Evidence and limitations
- 23:51Broader research context
- 28:31Analogy of solar system models
- 35:24Discussion and Q&A
related talks
32mWhy do LLMs hallucinate?
51mLarge Language Models: Friend, Foe, or Otherwise
37mLinear representations of concepts in modern AI models
35mReading the Mind of an LLM
40mThe Future of Language Models
40mIs my Large Language Model a Strange Loop?
52mGeneralization in the representations and computations of frontier language models.
41mTruth Checker: Generative Large Language Models and Hallucinations