Probabilistic Lipschitz Analysis of Neural Networks

on YouTube ↗19m

" summary

Defines probabilistic Lipschitzness: for pairs sampled from a generative input model, nearby inputs map to nearby outputs with high probability. Models inputs as a pcat program that samples a standard Gaussian then runs a generator g, analyzes f on g(S) by picking box-shaped S, computing S' = g(S), and proving k-Lipschitzness via a Jacobian analysis that bounds each partial derivative interval and over-approximates the operator norm. Gaussian mass of boxes is computed by z-table lookup. Empirically runs on MNIST and CIFAR show larger boxes and larger networks increase iteration time; code and proofs provided.

AI-written summary · about

" chapters (15)

1. 0:00Intro
2. 1:32Learning neural networks
3. 3:10Notions of correctness for neural networks
4. 4:16Why care about robustness?
5. 5:08Notions of robustness
6. 7:07Representing input distribution D
7. 8:13pcat language
8. 9:16Verifying probabilistic Lipschitzness
9. 10:23Lipschitzness via Jacobians
10. 11:37Jacobian analysis
11. 12:43Computing probabilistic mass
12. 13:50Some empirical results
13. 16:40Open questions
14. 17:51In the paper
15. 18:25Conclusion

" related talks

• 54mGraph Machine Learning & Random Graph ModelsSamuel Vaiter · Graph Learning Meets Theoretical Computer Science 2025
• 01h 03mLogic and Probabilistic Circuits 2YooJung Choi · Logic and Algorithms in Database Theory and AI Boot Camp
• 17mUniversality of Neural NetworksDan Mikulincer · Deep Learning Theory Symposium 2021
• 48mProvable Robustness Beyond Bound PropagationZico Kolter · Frontiers of Deep Learning 2019
• 01h 05mLogic and Probabilistic Circuits 1Guy Van den Broeck · Logic and Algorithms in Database Theory and AI Boot Camp
• 45mStudying Generalization in Deep Learning via PAC-BayesGintare Karolina Dziugaite · Frontiers of Deep Learning 2019
• 59mTutorial: Statistical Learning Theory and Neural Networks ISpencer Frei · Deep Learning Theory Workshop and Summer School 2022
• 01h 02mTutorial: Statistical Learning Theory and Neural Networks IISpencer Frei · Deep Learning Theory Workshop and Summer School 2022