Universality of Neural Networks

on YouTube ↗17m

" summary

Result gives a non-asymptotic central limit theorem for two-layer random-init networks, proving convergence of the random function law to a Gaussian process under a functional Wasserstein distance and a uniform metric for polynomials. Key tools are tensor power embedding (or harmonic-polynomial embedding), finite-dimensional CLTs and coupling arguments, and extensions beyond Gaussian weights. Polynomial activations yield explicit rates ~1/sqrt(K) in the stronger metric; general activations get weaker-metric bounds via polynomial approximation.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 0:39Setting
3. 2:22Background
4. 9:26Plan
5. 10:47The Embedding
6. 13:04A Central Limit Theorem for Neural Networks

" related talks

• 59mTutorial: Statistical Learning Theory and Neural Networks ISpencer Frei · Deep Learning Theory Workshop and Summer School 2022
• 49mComputation in Very Wide Neural NetworksYasaman Bahri · Frontiers of Deep Learning 2019
• 01h 04mRandomized Embeddings and Neural NetworksMert Pilanci · Sketching and Algorithm Design 2023
• 01h 14mRecent Developments in Over-parametrized Neural Networks, Part IIJason Lee · Deep Learning Boot Camp 2019
• 54mGraph Machine Learning & Random Graph ModelsSamuel Vaiter · Graph Learning Meets Theoretical Computer Science 2025
• 50mLearning and Generalization in Over-parametrized Neural Networks, Going Beyond KernelsYuanzhi Li · Frontiers of Deep Learning 2019
• 01h 02mTutorial: Statistical Learning Theory and Neural Networks IISpencer Frei · Deep Learning Theory Workshop and Summer School 2022
• 19mInterval Universal Approximation for Neural NetworksAws Albarghouthi, Zi Wang, Gautam Prakriya, Somesh Jha · POPL 2022