Pseudo-Labeling for Covariate Shift Adaptation

on YouTube ↗39m

" summary

Method splits labeled source into two parts to produce a set of candidate regressors and an independent imputation model with low bias, uses that imputer to create pseudolabels for model selection on unlabeled target examples, and proves a bias–variance decomposition showing low bias (not low variance) is crucial for faithful selection. Analysis for kernel ridge regression yields adaptivity driven by an effective sample size ne (e.g. ne ≈ np/b under bounded density ratio, ne ≈ sqrt(np) in a point-mass target).

AI-written summary · about

" chapters (8)

1. 0:00Pseudo-labeling introduction
2. 3:24Covariate shift challenges
3. 8:32Pseudo-labeling for model selection
4. 13:14Model selection theory
5. 18:08Kernel ridge regression adaptivity
6. 27:23Effective sample size examples
7. 33:15Summary of approach
8. 38:55Q&A

" related talks

• 42mBeyond Adversarial Approaches on Distributional RobustnessQi Lei · Adversarial Approaches in Machine Learning 2022
• 50mRobust Deep Learning Under Distribution ShiftZack Lipton · Frontiers of Deep Learning 2019
• 45mMulticalibration, Universal Adaptability and CausalityOmer Reingold · Algorithmic Aspects of Causal Inference 2022
• 38mMixed and Covariate-Dependent Graphical ModelsLiza Levina · Workshop on Succinct Data Representations and Applications (Simons Institute Big Data 2013)
• 45mThe Statistical Fairness-Accuracy FrontierAlireza Fallah · Learning from Heterogeneous Sources 2026
• 37mSelection bias: The elephant in the roomLucas Javier Bernardi · PyData Amsterdam 2018
• 44mActive Regression via Linear-Sample SparsificationEric Price · Real-Time Decision Making 2018
• 53mAdversarial Machine Learning and Instrumental Variables for Flexible Causal ModelingVasilis Syrgkanis · Adversarial Approaches in Machine Learning 2022