Beyond Adversarial Approaches on Distributional Robustness
summary
Proposes using label propagation over a learned representation to fix covariate shift: train on source labels, add consistency regularization over unlabeled target views, and propagate majority labels across connected components in representation space. Theory assumes an expansiveness property of augmentations, shows recovery when the teacher is 51% per component and gives a target-error upper bound roughly mu/C error bound. Representation comes from similarity-based self-supervised learning (analyzed via nonlinear CCA). Empirical gains beat MDD on OfficeHome/Office31 and BREEDS subpopulation-shift benchmarks.
AI-written summary · about
chapters (18)
- 0:00Intro
- 1:09What Went Wrong?
- 2:02Outline
- 2:42Types of Distribution Shifts
- 4:14Covariate shift
- 5:24Existing Method and Their Success Scenarios
- 8:20Our New Framework: Subpopulation Shift
- 12:19Subpopulation on the Representation Space
- 14:43Algorithmic Framework
- 19:06Beyond Perfect Case
- 21:05Main theorem
- 22:09Generalized Subpopulation Shift
- 25:08Experiments: Subpopulation Shift Dataset
- 25:58Self-supervised Learning
- 27:30Connection to CCA
- 29:00Main Result 2: Linear separation with new features from CCA
- 30:58Conclusion
- 36:52Warm-Up: Ideal Case
related talks
50mRobust Deep Learning Under Distribution Shift
39mPseudo-Labeling for Covariate Shift Adaptation
49mA New Perspective on Adversarial Perturbations
34mDistributional Robustness, Learning, and Empirical Likelihood
38mRepresentation via Representations: Improving Generalization to Unseen Populations
01h 01mDistribution Shift as Underspecification, and What We Might Do About It
37mDomain Adaptation-- 20 years of theory chasing practice
12mDistribution-Free Robustness