Domain Adaptation-- 20 years of theory chasing practice
summary
Describes a 2005 NLP fix that represents each token by a pivot-word window embedding and then trains a linear classifier on Wall Street Journal labels that surprisingly transfers to medical text without target labels. Theory formalizes that success with uniform-convergence bounds: target risk ≤ source risk + marginal discrepancy + labeling-shift (lambda), and shows those discrepancy terms are estimable for handcrafted embeddings. Modern learned embeddings and foundation models break that transparency, and there is a provable no-free-lunch change-detection barrier: detecting or measuring shift needs prior assumptions.
AI-written summary · about
chapters (5)
- 0:00Introduction to DA
- 6:08Theoretical foundations
- 15:52Change detection challenge
- 17:41Practical research evolution
- 28:40Future and conclusion
related talks
42mBeyond Adversarial Approaches on Distributional Robustness
50mRobust Deep Learning Under Distribution Shift
41mFrom Embeddings To Index: A Practitioner's Guide To Domain-Adapted Neural Search
01h 01mDistribution Shift as Underspecification, and What We Might Do About It
22mMaking Natural Language Processing Robust to Sociolinguistic Variation
31mKeynote Q&A – Barbara Plank
01h 10mModern paradigms of generalization, the heliocentric model of Aristarchus,...
31mAttacking Deep Learning-Based NLP Systems