The Prefrontal Cortex as a Meta-Reinforcement Learning System
summary
Claim: prefrontal cortex implements a learned learning algorithm — a meta‑reinforcement learner — realized as activity dynamics rather than ongoing synaptic updates. Evidence: trained LSTM recurrent net with a convolutional front end using A3C training and backprop through time on thousands of bandit and visual two‑choice tasks. Surprising results: the network matches Harlow‑style rapid one‑trial learning, outperforms Thompson/UCB on regret, reproduces Behrens volatility‑adaptive learning rates and the inferred value effect in dopamine signals. An episodic memory add‑on handles longer delays.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 2:18Why is it relevant
- 4:07Recurrent Neural Networks
- 4:59Neuroscience and AI
- 5:55Brendan Lake and Atari
- 8:17Brendans Background Knowledge
- 9:06Learning to Learn
- 12:18Learning vs Inference
- 16:01Multilayer Perceptron
- 17:51Recurrent Neural Network
- 18:25Bandit Problems
- 28:43Harlows Task
- 29:57Two Neuroscience
- 31:17Hippocampal Amnesia
- 32:04Neuroscience
- 45:07Inferred Value Effect
- 45:59Other Simulations
- 46:09Summary
- 47:46virtuous circle
- 48:00collaborators
related talks
50mFlexible Neural Networks and the Frontiers of Meta-Learning
28mDeep Reinforcement Learning: theory, intuition, code
27mWhat can artificial neural networks learn from biological neuromodulatory systems?
41mArtificial Intelligence: Let's Think About Learning
14mHuman-in-the-Loop Reinforcement Learning
49mML: Teaching machine to learn from its own mistakes (...)
32mHow Neuroscience Makes AI Intelligent
01h 31mReinforcement Learning- Pac-Man