Tensor Decompositions for Estimating Latent Variable Models
summary
Claims that tensors yield closed‑form, polynomial‑time estimators for several latent models under mild nondegeneracy: use the method of moments, whiten the second moment, then perform an orthogonal tensor decomposition on the third moment (size reduced to K^3) to recover K topic/mean vectors; identifiability for topic models needs document length three, HMMs need window size about log_D K. Algorithm uses a power method with restarts, a perturbation/Davis‑Kahan analog, sample complexity tied to the smallest eigenvalue; works in practice and gives provable robust error bounds.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:26Two canonical examples
- 2:05What are the limits of learning?
- 6:52A Different Approach
- 7:54Related Work
- 8:31Mixture Models
- 12:06The Method of Moments
- 14:59vector notation and multinomials!
- 18:06With the second moment?
- 21:46With three words per document?
- 26:08Tensors and Linear Algebra
- 27:04Tensor eigenvectors
- 27:58Estimation
- 32:10Algorithm: Tensor Power Iteration
- 42:31Mixtures of spherical Gaussians Theorem
- 47:26The Minimal Realization Problem for HMMs
related talks
56mTensor Decompositions for Learning Latent Variable Models I
53mTensor Decompositions for Learning Latent Variable Models II
52mTensor Methods for Learning Latent Variable Models: Theory and Practice
01h 03mRecent Progress in High-Dimensional Learning
01h 11mTensor Decompositions: Uniqueness and Smoothed Analysis
49mSpectral Algorithms for Learning Latent Variable Models
54mLearning Overcomplete Latent Variable Models through Tensor Power Method
52mAverage-Case Overcomplete Tensor Decomposition