An Integrated Cognitive Architecture
summary
Builds agents that combine a content-addressable external memory (originally Differentiable Neural Computer / Neural Turing Machine) with learned latent state predictors using variational autoencoders and a Gluck & Myers–style return reconstruction loss. Uses truncated BPTT with K=20 (~1.3-second truncated BPTT) while storing many state codes to memory, then trains a policy that reads those codes. The resulting Merlin (Memory Inference and Reinforcement Learning Agent) solves partial‑observability tasks, one‑shot reward evaluation, latent learning and six‑minute memory gaps that break ordinary gradient or end‑to‑end RL.
AI-written summary · about
chapters (28)
- 0:00Intro
- 1:58Talk Outline
- 3:08Neural Turing Machines (NTMs) and Differentiable Neural Computers (DNCs)
- 5:07A Simple DNC
- 12:07Learning Complex Data Structures (Random Graphs)
- 13:46Tube Traversal Problem
- 15:41Reinforcement Learning
- 17:04Challenge (Memory Game)
- 17:51Control Architecture "a"
- 21:10Policy Gradient
- 22:36State Estimation
- 23:12in a Goal-Directed Agent
- 23:29Unsupervised Learning
- 26:38Unsupervised Predictive Modeling
- 27:08What the Mechanism Looks Like
- 28:25Memory-Based Predictor
- 31:44MERLIN on the Memory Game
- 33:02Scaling Up
- 35:02Truncation for Very Long Tasks
- 35:55Navigation
- 38:05Return Prediction
- 38:47Gluck and Myers
- 40:56Hierarchical Behavior
- 42:46Arbitrary Visuomotor Mapping (Wise and Murray, 2000)
- 43:08Rapid Reward Valuation (One- Shot)
- 43:30An Episodic Task that Breaks Gradient Flow
- 44:12Latent Learning
- 45:54Conclusions and Prospects
related talks
15mMemory-Augmented AI Agents: Enhancing Long-Term Reasoning and Context Awareness
46mThe Anatomy of Memory in Humans and AI Agents
58mAdvanced Autonomous AI Agents: Memory, Collaboration and Evolution in Action
42mMaking state management intelligent
32mThe AI Operating System: Stateful Agents with Letta
01h 49mBuilding Memory Augmented Agents with MemoRizz
23mYour Agent Has Amnesia: Why Mobile AI Agents Need Memory to Be Useful
24mMulti-Modal AI Agents Explained | Open-Vocabulary Alignment