Dreadful Frailties in Propensity Score Matching and How to Fix Them
summary
Demonstrates that using standardized mean difference alone yields huge variability across ~100 propensity/matching pipelines, shows synthetic tests where selecting by a new A2A metric (run over many artificial splits) cuts ATE variability dramatically—reporting up to a 90% reduction in some setups—and moves a Groupon case from ~1,000 to ~600 range. Provides automated pipeline and code in popmatch package, warns metric is ordinal, computationally heavy, but often selects methods with lower true error.
AI-written summary · about
chapters (13)
- 0:00Propensity score challenges
- 1:07Predictive maintenance in healthcare
- 3:13Propensity score bias
- 5:20Standard PSM pipeline
- 7:31Validation rules for submissions
- 9:15Problems with validation
- 10:40Introducing A2A metric
- 13:09Automating A2A computation
- 14:35Evaluating with synthetic data
- 16:03Validation strategies compared
- 17:48Real-world results and limits
- 18:32Conclusion and future work
- 22:42Q&A
related talks
38mTracking and Improving Information in the Service of Fairness
01h 31mHow To Measure And Mitigate Unfair Bias in Machine Learning Models
46mThe Measure and Mismeasure of Fairness
27mUnmeasured Spatial Confounding
28mMeasuring Model Fairness
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
30mMachine Learning Techniques for Class Imbalances & Adversaries
31mPython and the Holy Grail of Causal Inference