Towards Biologically Plausible Deep Learning: ...
summary
Leaky‑integrator neurons are shown to perform Langevin MCMC inference on an energy‑based model whose update direction equals the energy gradient, and the first few timesteps of settling from a fixed point reproduce layerwise gradients equivalent to backpropagation; weight changes map to STDP‑like weight updates and a Hebbian equilibrium‑propagation recipe is implemented as equilibrium propagation, trained to zero error on MNIST but with lower efficiency than standard backprop. Simulations and analytic Taylor approximations support these claims.
AI-written summary · about
chapters (18)
- 0:00Intro
- 2:34Stochastic Gradient Descent (SGD) Training
- 3:38Biological Implausibility of Backpropagation
- 4:49Leaky Integrator Neurons
- 6:40An Energy-Based Model
- 7:41Leaky Integration as Langevin MCMC
- 11:08The Energy Function
- 13:04Equilibria of the Dynamics
- 13:59Input, Output and Hidden Units
- 15:32Predicting the Output
- 16:28Propagation of Perturbations
- 21:30Outline
- 23:34The Wight Update
- 25:12Spike-Timing-Dependent Plasticity
- 27:37Rate based STDP-approximation
- 28:42Equilibrium Propagation
- 30:34Summary
- 31:48References
related talks
32mHow Neuroscience Makes AI Intelligent
29mDeep Learning, Neuroscience and the future of AI
57mTutorial on Deep Learning III
31mInterpreting Machine Learning From the Perspective of Nonequilibrium Systems
41mArtificial Intelligence: Let's Think About Learning
47mInterpreting Deep Neural Networks (DNNs)
01h 10mDemystifying Deep Learning for Data Scientists
27mWhat can artificial neural networks learn from biological neuromodulatory systems?