Causally motivated robustness to shortcut learning

on YouTube ↗45m

" summary

Claims that causally-motivated regularization—combining importance weighting to mimic an “ideal” uncorrelated distribution with an MMD penalty that enforces independence—reduces shortcut learning, gives finite-sample efficiency (smaller Rademacher bounds) and yields better robustness than L2 baselines under a range of simulated shifts. Demonstrated on chest X‑ray and semi‑synthetic bird‑background data, ablations show both reweighting and MMD are necessary and the combined method consistently outperforms the comparator across tested correlations.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 1:23Scaling model costs
3. 3:11Sentiment classification shortcuts
4. 9:09Pneumonia detection assumptions
5. 16:57Causal DAGs and pathways
6. 21:38Relaxing distribution assumptions
7. 28:21Reweighting distributions
8. 31:57Robustness on bird backgrounds
9. 37:54Discovering shortcuts
10. 39:42Causality beyond prediction
11. 40:43Q&A

" related talks

• 34mDistributional Robustness, Learning, and Empirical LikelihoodJohn Duchi · Optimization, Statistics and Uncertainty
• 53mAdversarial Machine Learning and Instrumental Variables for Flexible Causal ModelingVasilis Syrgkanis · Adversarial Approaches in Machine Learning 2022
• 44mInvariance, Causality and Novel RobustnessPeter Bühlmann · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 12mDistribution-Free RobustnessSitan Chen · Meet the Fellows Welcome Event
• 45mMulticalibration, Universal Adaptability and CausalityOmer Reingold · Algorithmic Aspects of Causal Inference 2022
• 42mBeyond Adversarial Approaches on Distributional RobustnessQi Lei · Adversarial Approaches in Machine Learning 2022
• 43mRobustness in High-Dimensional Inference TasksJelena Bradic · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 09mTowards Credible and Effective Data-Driven Decision-MakingAngela Zhou · Meet the Fellows Welcome Event