Anima Anandkumar, Electrical Engineering and CS Dept, UC Irvine
summary
Presents a concrete recipe: form higher‑order moments, run SVD on pairwise moments with random projections to whiten, then perform tensor decomposition via tensor contractions; this yields provable global recovery for mixtures, HMMs, ICA, topic models and even first‑layer neural network weights. Implements on BLAS/GPU kernels and a Spark implementation, scales linearly and in practice is >100x faster than variational inference on a workstation, with distributed code and empirical speedups shown.
AI-written summary · about
chapters (22)
- 0:00Start
- 2:56Learning with Big Data
- 4:29Optimization for Learning
- 5:22Convex vs. Nonconvex Optimization
- 8:27Outline
- 9:49Tensor Representation of Higher Order Moments
- 11:11Spectral Decomposition of Tensors
- 12:14Tensor Decomposition Methods
- 13:03Summary on Tensor Methods
- 13:27Preliminary Results on Spark
- 13:55Tensor Methods for Learning LVMs
- 15:18Overview of Our Approach
- 17:52Community Detection on Yelp and DBLP datasets
- 18:44Tensor Methods for Training Neural Networks
- 18:55Toy Example: Failure of Backpropagation
- 19:16Backpropagation vs. Our Method
- 19:38Feature Transformation for Training Neural Networks
- 19:58Score Function Transformations
- 20:20Reinforcement Learning (RL) of POMDP
- 20:51Convolutional Tensor Decomposition
- 21:02Hierarchical Tensor Decomposition
- 21:59My Research Group and Resources
related talks
33mAnimashree Anandkumar, Electrical Engineering and CS Dept, UC Irvine @ MLconf SEA
52mTensor Methods for Learning Latent Variable Models: Theory and Practice
30mHanie Sedghi, Research Scientist, Allen Institute at MLconf Seattle 2017
53mTensor Decompositions for Learning Latent Variable Models II
49mSpectral Algorithms for Learning Latent Variable Models
01h 02mTheory and Big Data
58mTensor Decomposition, Sparse Representations and Applications
01h 31mUnsupervised Learning