TBD

on YouTube ↗45m

" summary

Proposes a statistical IRM scaffold: three concrete conditions—source-preservation, contraction, Bayes-unifies—and proves that with an oracle representation phi a k‑nearest‑neighbor classifier on source samples reaches target Bayes risk; unlabeled target samples rule out representations that fail contraction, while a small amount of labeled target data disambiguates remaining candidates with sample complexity tied to the VC class of pairwise distance tests. For imbalanced classes, uses convex‑hull geometry and extreme‑value theory (Fisher‑Tippett‑Gnedenko) to show subsampling helps for Gumbel vs Weibull tails* because exponential tails push the max‑margin boundary, whereas bounded‑tail cases don’t benefit.

AI-written summary · about

" chapters (21)

1. 0:00Introduction
2. 0:44Distribution shift in ML theory
3. 2:29Practical distribution shift examples
4. 5:47Spurious correlations framework
5. 7:14Invariant risk minimization and its limitations
6. 8:58Building statistical IRM framework
7. 12:14Representation learning assumptions
8. 16:44IRM assumptions explained
9. 19:44Theoretical guarantees with oracle phi
10. 22:14Finding phi without oracle
11. 24:41Role of unlabeled target data
12. 27:14Insufficient source/target data
13. 29:01Spurious correlation results summary
14. 30:30Imbalanced data distribution shift intro
15. 31:54Practitioner approaches to imbalance
16. 34:20Max-margin classifier properties
17. 36:48Extreme value theory for imbalance
18. 38:14Toy example for imbalanced classes
19. 40:44Gumbel vs Weibull distributions
20. 42:33Conclusion and open questions
21. 43:35Q&A

" related talks

• 42mDistribution-Specific Analysis of Nearest Neighbor Search and ClassificationSanjoy Dasgupta · Learning, Algorithm Design and Beyond Worst-Case Analysis
• 51mRandom Projections for Probabilistic InferenceStefano Ermon · Uncertainty in Computation 2016
• 01h 23mGeneralization IIPeter Bartlett, Sasha Rakhlin · Deep Learning Boot Camp 2019
• 34mDistributional Robustness, Learning, and Empirical LikelihoodJohn Duchi · Optimization, Statistics and Uncertainty
• 01h 02mTheory and Big DataRavi Kannan · Big Data Boot Camp 2013
• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022
• 01h 18mGeneralization IIIPeter Bartlett, Sasha Rakhlin · Deep Learning Boot Camp 2019
• 53mDesigning Robust LearnersJacob Steinhardt · Emerging Challenges in Deep Learning (Simons Institute workshop, 2019)