Optimally Weighted PCA for High-Dimensional Heteroscedastic Data
Talkat Robust and High-Dimensional Statistics (Simons Institute workshop, 2018) · 2018-10-29..11-02·
summary
Asymptotic analysis shows column-weighted PCA recovers a low-rank subspace better than unweighted PCA under heteroscedastic noise: with n,d → ∞ at fixed ratio there is a phase transition and concrete recovery formulas from random-matrix limits. The optimal column weight is explicit and depends on noise variance and signal amplitude, roughly weights = 1/sigma^2 × 1/(theta^2+sigma^2), sitting between inverse-variance and inverse-variance-squared. Simulations (e.g. 20% clean, 80% noisy; theta≈1.5) confirm improved, sometimes near-binary downweighting of very noisy samples.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:09Modern data: High-Dimensional & Heteroscedastic
- 4:44Existing results for (unweighted) PCA
- 9:23Weighted PCA: Dimensionality Reduction
- 10:36Main Question: How should we weight?
- 12:36An illustrative example with a sneak peek
- 15:59Theory: Asymptotic Weighted PCA performance
- 19:06Theory: inverse noise variance
- 21:07Asymptotic results compared to finite simulation
- 22:10Asymptotically Optimal Weights
- 24:48Sub-optimal weighting: two case studies
- 27:34Impact of contamination on component recovery
- 28:44Impact of noise variances on component recovery
- 29:51Impact of adding noisy data on component recovery
- 32:02Discussion: Maximum Likelihood
- 34:34Conclusion
related talks
49mRobust PCA via Non-convex Methods: Provable Bounds
52mIndependent Component Analysis: From Theory to Practice and Back
43mSpectral Approaches to Nearest Neighbor Search
45mHigh Dimensional Robust Sparse Regression
39mMemory Limited, Streaming PCA
01h 30mHigh-Dimensional Statistics I
01h 31mUnsupervised Learning
30mHanie Sedghi, Research Scientist, Allen Institute at MLconf Seattle 2017