An overview of classical robust statistics and generalizations to the future

on YouTube ↗01h 15m

" summary

Huber theory is presented with precise claims: Huber loss is minimax-optimal for asymptotic variance under Huber ε‑contamination (γ is tuned to ε), the M‑estimator variance depends on ψ = l′ and the influence function proportional to ψ explains why bounded ψ controls robustness. Nonconvex “redescending” losses give finite rejection points. For heterogeneous one‑sample‑per‑distribution data, modal‑interval and shorth estimators with k ≈ log n succeed (classical cube‑root n−1/3 rates appear for other regimes). Private testing uses a clamped log-likelihood ratio per sample plus Laplace noise and is minimax‑optimal up to constants.

AI-written summary · about

" chapters (13)

1. 0:00Introduction to robustness
2. 2:09M-estimators for location
3. 5:28Huber contamination model
4. 9:02Understanding M-estimators
5. 13:38Minimax variance optimality
6. 19:38Huber's saddle point solution
7. 28:55Influence function framework
8. 35:08Robustness of Huber estimator
9. 40:41Redescending M-estimators
10. 44:24Modern M-estimator settings
11. 47:37Modal and shortest gap methods
12. 52:16Asymptotics of unconventional estimators
13. 56:53Robust testing and privacy

" related talks

• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022
• 12mDistribution-Free RobustnessSitan Chen · Meet the Fellows Welcome Event
• 29mOn Learning Distributions from their SamplesSudeep Kamath · Information Theory, Learning and Big Data (Simons Institute, 2015)
• 53mDesigning Robust LearnersJacob Steinhardt · Emerging Challenges in Deep Learning (Simons Institute workshop, 2019)
• 34mDistributional Robustness, Learning, and Empirical LikelihoodJohn Duchi · Optimization, Statistics and Uncertainty
• 41mEfficient Algorithms for Outlier-Robust RegressionAdam R. Klivans · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 44mA Modern Maximum-Likelihood Theory for High-Dimensional Logistic RegressionPragya Sur · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 48mComputational-Statistical Tradeoffs in Robust High-Dimensional EstimationIlias Diakonikolas · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)