Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures
summary
Presents Gaussian Semantics: approximate a bounded probabilistic program’s exact distribution by Gaussian mixtures whose parameters are set by matching moments at each program location. Proves a universal approximation theorem and implements an analytic second-order Gaussian approximation (SOGA / SOA) that matches means and covariances exactly, returns posterior means and covariances, and avoids costly integrals or sampling. Scales well on collaborative filtering and mixed discrete–continuous benchmarks, often beating Stan MCMC, Pyro VI and symbolic PSI on accuracy and runtime, but does not handle unbounded loops.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 2:21Probabilistic programs as transformers
- 3:45Inference problem and methods
- 4:27Gaussian mixtures contribution
- 6:14Why Gaussian mixtures fit
- 8:00Gaussian semantics with moment matching
- 10:11Moment matching operator
- 11:38Higher-order moments challenges
- 12:19SOA tool and approximations
- 13:22Experimental evaluation
- 14:27Applications in mixed models
- 15:29Future work
- 16:31Q&A
- 17:35Q&A
- 18:18Q&A
- 19:28Q&A
related talks
35mLearning General Gaussian Mixtures With Efficient Score Matching
44mMixture Models, Robustness, and Sum of Squares Proofs
09mConvex Programs, Computational Phase Transitions, and Robust Algorithms
52mLearning Mixture of Gaussians using Moment Tensor
29mApproximating Spherical Gaussian Mixtures
21mGuaranteed inference for probabilistic programs: a parallelisable, small-step o...
58mUniversality of Approximate Message Passing on Semi-random Matrices
20mProbabilistic Programming with Stochastic Probabilities