The Importance of Forgetting for Both Humans & Machines
summary
Forgetting is framed as an adaptive mechanism supported by experiments: rats in a water maze or smell‑tray tasks that run on wheels to boost neurogenesis relearn faster after the reward moves, and reinforcement‑learning agents that “forget” adapt quicker when environments change. Production ML uses retrain‑on‑schedule or monitor‑triggered retraining and MLOps tooling like TFX with a metadata store to track pipelines. Large foundational models (e.g. 175 billion parameters trained on 45 TB at ~\$12M) cannot be retrained frequently, so ensembles and mixtures‑of‑experts handle drift.
AI-written summary · about
chapters (11)
- 0:00Intro
- 0:24Remember, learn, forget
- 8:15Testing the hypothesis
- 10:19What about ML?
- 16:13How do models forget?
- 18:48MLOps
- 20:57Training & deploying models
- 25:11What about tomorrow?
- 29:11Learning from studying our brains
- 31:03Resources
- 31:20Outro
related talks
30mUnforgettable, that's what you are: Evaluating Machine Unlearning and Forgetting
28mWhy we do ML model retraining wrong, and how to do better
31mGetting AI to Forget About You: Data Deletion From Machine Learning Models
27mMemory Engineering: Going Beyond Context Engineering
40mProductionizing Machine-Learning Services: Lessons from Google SRE
10mAvoiding the Oops with MLOps
23mHow can a learnt ML model unlearn something: Framework for "Machine Unlearning"
46mThe Anatomy of Memory in Humans and AI Agents