Can Cognitive Neuroscience Provide a Theory of Deep Learning...

on YouTube ↗43m

" summary

Claims that biological brains limit multitasking because shared, reused representations cause interference lead to concrete measurements: convert tasks to an interference graph and show the maximum independent set bounds parallel capacity; simulations train 20 randomly initialized networks on 16 one-hot tasks and find initialization controls representation reuse and interference; even modest cross‑connectivity in a 50‑node model cuts parallel capacity by orders of magnitude; plans extend to weighted graphs, full deep nets and MRI validation against human data.

AI-written summary · about

" chapters (29)

1. 0:00Start
2. 11:11CRACKS APPEAR, DISRUPTIVE IDEAS 30 YEARS ON
3. 11:46COGNITIVE NEUROSCIENCE
4. 12:58NEURAL NETWORK PRELIMINARIES
5. 16:53ARBITRARY FUNCTIONS
6. 18:23THE ORIGINAL TENETS OF PARALLEL DISTRIBUTED PROCESSING
7. 21:02BRAIN-INSPIRED MACHINE LEARNING
8. 22:12WINNING TOP SPOTS IN VISUAL RECOGNITION CHALLENGES
9. 23:16STACKED ATTENTION NETWORKS FOR IMAGE QUESTION ANSWERING
10. 23:52WHAT ARE SITTING IN THE BASKET ON A BICYCLE?
11. 25:49REPRESENTATION ENCODING: MEANINGFUL AND CONSISTENT
12. 26:50CONVERGENT LEARNING?
13. 27:46THE APPEAL OF HIGHLY-PARALLEL NEURAL NETWORKS
14. 29:28THE PARADOX - ONE TASK AT A TIME
15. 29:50A FUNDAMENTAL PUZZLE CONCERNING HUMAN PROCESSING
16. 31:16MULTI-TASKING AND CROSS-TALK
17. 33:18A DEMONSTRATION OF INTERFERENCE
18. 33:38MULTI-TASKING INTERFERENCE (IN THE STROOP TEST)
19. 34:51CONTROL-DEMANDING BEHAVIOR (FENG ET AL. 2014)
20. 35:36TYPES OF INTERFERENCE
21. 35:55MAXIMUM INDEPENDENT SET (MIS)
22. 36:53NETWORK STRUCTURE (DISTRIBUTION COMPLEXITY)
23. 37:32CONSTRAINTS ON PARALLEL PROCESSING CAPABILITY
24. 38:37TRADE-OFF BETWEEN GENERALIZATION AND PARALLELISM: FEEDFORWARD NETWORKS
25. 38:50TRAINING/TEST DETAILS
26. 39:37SHARED REPRESENTATIONS
27. 40:46GENERALIZATION VS PARALLEL PROCESSING CAPABILITY
28. 40:56PARALLEL PROCESSING CAPABILITY VS MAX INITIAL WEIGHT
29. 41:10FUTURE WORK

" related talks

• 45mMachine Learning, Deep Networks and Interpretability in Systems, Cognitive and...Jack Gallant · Interpretable Machine Learning in Natural and Social Sciences (2022)
• 47mCapacity of Neural Networks for Lifelong Learning of Composable TasksLes Valiant · Computational Theories of the Brain 2018
• 32mHow Neuroscience Makes AI IntelligentDalton A. R. Sakthivadivel · PyData Austin 2019
• 01h 14mTissue vs silicone musings on the future of deep learning hardware and softwareNir Shavit · Hydra 2020
• 26mNetwork neuroscience: where the brain and social networks works the sameIlya Zakharov · DSC Europe 23
• 01h 03mPlenary Discussion: What Is Missing in Current Theories of Brain Computation?Computational Theories of the Brain 2018
• 36mBrain's Neural Network as a serviceDhivya Raj · Converge 2022
• 57mWhat Should a Computational Theory of Cortex Explain?Leslie Valiant · Symposium on Visions of the Theory of Computing 2013