Tensor Decompositions for Learning Latent Variable Models II

on YouTube ↗53m

" summary

Shows that third-order moments plus a whitening reduction recover latent mixtures when component means are linearly independent and weights are bounded away from zero, using tensor decompositions (Jennrich/GD-style or greedy rank‑one deflation). Empirical moments converge at ~1/sqrt(N); orthogonal reduction makes components identifiable and algorithms error‑tolerant for noise ε (needs ε ≲ 1/poly(K)); deflation errors shrink like ε^2 in orthogonal directions. Also covers tensor power iteration (quadratic local convergence) and extensions to multi‑view models, HMMs and overcomplete cases via higher‑order moments.

AI-written summary · about

" chapters (23)

1. 0:00Learning algorithms
2. 0:35Moment decompositions
3. 2:11Moments for spherical Gaussian mixtures
4. 3:52Use of moments for mixtures of spherical Gaussians
5. 6:50Third-order moments of spherical Gaussian mixtures
6. 8:51Tensor decomposition for spherical Gaussian mixtures
7. 11:08Even more Gaussian matures
8. 14:03Use of moments for mixtures of linear regressions
9. 15:47Simpler setting mixed random linear equations
10. 17:02Multi-view interpretation of topic model
11. 20:09Examples of multi-view mixture models
12. 23:16Moment estimates
13. 24:25Using Jennrich's algorithm
14. 25:27Stability of Jennrich's algorithm
15. 26:52Reduction to orthonormal case
16. 29:14Nearly orthogonally decomposable tensors d-K
17. 31:26Exact orthogonally decomposable tensor
18. 34:14Rank-1 approximation problem
19. 36:15Fixed-point algorithm
20. 39:28Comparison to matrix power iteration
21. 45:14Errors from deflation
22. 46:49Effect of deflation errors
23. 49:55Summary

" related talks

• 56mTensor Decompositions for Estimating Latent Variable ModelsSham Kakade · Tensors in Computer Science and Geometry 2014
• 52mTensor Methods for Learning Latent Variable Models: Theory and PracticeAnimashree Anandkumar · Spectral Algorithms: From Theory to Practice 2014
• 01h 03mRecent Progress in High-Dimensional LearningAnkur Moitra · Probability, Geometry, and Computation in High Dimensions Boot Camp (Simons Institute, Fall 2020)
• 49mSpectral Algorithms for Learning Latent Variable ModelsSham Kakade · Spectral Algorithms: From Theory to Practice 2014
• 54mLearning Overcomplete Latent Variable Models through Tensor Power MethodRong Ge · Semidefinite Optimization, Approximation and Applications 2014
• 01h 11mTensor Decompositions: Uniqueness and Smoothed AnalysisMoses Charikar · Semidefinite Optimization, Approximation and Applications 2014
• 52mLearning Mixture of Gaussians using Moment TensorRong Ge · Tensors in Computer Science and Geometry 2014
• 58mTensor Decomposition, Sparse Representations and ApplicationsBernard Mourrain · Tensors in Computer Science and Geometry 2014