Scale Calibration for High-Dimensional Robust Regression

on YouTube ↗47m

" summary

Proposes a practical pipeline that recovers sparse high-dimensional regression under heavy tails without knowing error scale: convex Huber+L1 initialization gives a provable basin, then optimize a (possibly nonconvex) robust loss and apply a one-step Newton update to regain asymptotic efficiency. Scale is chosen by computing estimators on a grid of scales and using pairwise screening (Lepskiʼs method) to pick a “good” set; theory shows local and global optima near beta* under restricted strong convexity, but relies on IID assumptions and RE constants.

AI-written summary · about

" chapters (12)

1. 0:00Intro
2. 0:29High-dimensional linear regression
3. 1:00"Robust" M-estimators
4. 5:39When the scale is unknown...
5. 8:59Some proposals
6. 20:10Some optimization theory (L. 17)
7. 22:20Motivating calculation
8. 25:09Technical challenges
9. 29:54Lepski's method
10. 35:23Statistical guarantee
11. 36:16Improving efficiency
12. 39:46Other adaptable problems?

" related talks

• 45mHigh Dimensional Robust Sparse RegressionConstantine Caramanis · Sublinear Algorithms and Nearest-Neighbor Search
• 44mActive Regression via Linear-Sample SparsificationEric Price · Real-Time Decision Making 2018
• 01h 23mImplicit Regularization IINati Srebro · Deep Learning Boot Camp 2019
• 46mKernel and Deep Regimes in Overparameterized LearningSuriya Gunasekar · Frontiers of Deep Learning 2019
• 49mFast Algorithms for Regression ProblemsDeeksha Adil · Optimization and Algorithm Design 2023
• 01h 30mHigh-Dimensional Statistics IIMartin Wainwright · Big Data Boot Camp 2013
• 01h 01mRobustness, Stochastics, Uncertainty 2Sasha Rakhlin, Ben Recht, Laurent El Ghaoui · Bridging Continuous and Discrete Optimization Boot Camp 2017
• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022