Transfer learning via local convergence rates of the nonparametric least squares estimator

on YouTube ↗41m

" summary

Shows that the empirical risk minimizer over 1‑Lipschitz functions attains minimax local rates under covariate shift by proving a pointwise bound tn defined implicitly: mass of (x ± tn) times tn^2 ≍ (log n)/n. Assumes a doubling condition on the source density, gets the usual n−2/3 when density is bounded away from zero, but slower rates in low‑density regions governed by an exponent α (loss occurs when α > 3/2). Proof uses a contradiction and a careful perturbation construction; combined estimators with m target samples need m ≫ n^{3/(3+α)} to restore n−2/3.

AI-written summary · about

" chapters (12)

1. 0:00Covariate shift in nonparametric regression
2. 1:49Covariate shift challenges for least squares
3. 3:57Lipschitz functions relevance
4. 7:10Local convergence rates under low design
5. 12:30Why standard approaches fail
6. 17:52Proof strategy: contradiction and perturbation
7. 21:50Implications of local rate results
8. 23:38Covariate shift example with density decay
9. 28:10Combining source and target samples
10. 33:11Extensions and open problems
11. 35:38Summary and future directions
12. 39:39Discussion and audience questions

" related talks

• 01h 30mHigh-Dimensional Statistics IIMartin Wainwright · Big Data Boot Camp 2013
• 01h 22mGeneralization IVPeter Bartlett, Sasha Rakhlin · Deep Learning Boot Camp 2019
• 01h 23mImplicit Regularization IINati Srebro · Deep Learning Boot Camp 2019
• 29mOn Learning Distributions from their SamplesSudeep Kamath · Information Theory, Learning and Big Data (Simons Institute, 2015)
• 27mInformation Measure Estimation and Applications: Boosting the Sample Size from n to n log nJiantao Jiao · Information Theory, Learning and Big Data (Simons Institute, 2015)
• 53mOptimality in Locally Private Estimation and LearningJohn Duchi · Privacy and the Science of Data Analysis 2019
• 58mFirst-Order Stochastic OptimizationRachel Ward · Foundations of Data Science Boot Camp 2018
• 33mTradeoffs in Large Scale Learning: Statistical Accuracy vs. Numerical PrecisionSham Kakade · Information Theory, Learning and Big Data (Simons Institute, 2015)