TAP Free Energy, Spin Glasses, and Variational Inference

on YouTube ↗45m

" summary

For the spiked Wigner model with ±1 prior, noise variance 1/N and signal strength λ (constant, λ>λ0), the usual product mean-field fails to approximate Bayes posterior: any mean-field critical point stays a nonzero distance from the Bayes estimator. Adding the Onsager correction gives the TAP free energy with stationarity m = tanh(λYm − λ^2(1−Q)m) and correction term (λ^2/4)(1−Q)^2. The paper proves low-energy critical points (energy < −λ^2/3) recover the Bayes risk Xi(λ) as N→∞, using Kac-Rice counting of critical points and a complexity function.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 6:35Variational Inference
3. 14:55TAP
4. 20:55posterior distribution
5. 25:45approximation
6. 30:00probability distribution
7. 32:50proof

" related talks

• 01h 06mTutorial: Methods from Statistical Physics IIAhmed El Alaoui · Deep Learning Theory Workshop and Summer School 2022
• 32mLocal Convexity Of The Tap Free Energy And Amp Convergence For Z2-SynchronizationSong Mei · Rigorous Evidence for Information-Computation Trade-offs
• 58mAn Instability in Variational Methods for Learning Topic ModelsAndrea Montanari · Optimization, Statistics and Uncertainty
• 36mTo Be (Hard) or Not Be:...Florent Krzakala · Rigorous Evidence for Information-Computation Trade-offs
• 59mTutorial: Methods from Statistical Physics IAhmed El Alaoui · Deep Learning Theory Workshop and Summer School 2022
• 01h 18mStatistical Physics and Computation in High DimensionFlorent Krzakala, Lenka Zdeborova · Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020)
• 01h 08mTutorial: Methods from Statistical Physics IIIAhmed El Alaoui · Deep Learning Theory Workshop and Summer School 2022
• 49mApproximate Message Passing Algorithms For Orthogonally Invariant ModelsZhou Fan · Algorithmic Advances for Statistical Inference with Combinatorial Structure