Discovering Unwarranted Associations in Data-Driven...
summary
FairTest is a Python toolkit integrated into SciPy that automatically finds statistically significant unwarranted associations by treating an application as a black box and running an association-guided decision tree over a training/validation split. It supports metrics like mutual information, correlation and logistic regression, ranks findings to reduce false positives, and reports contextual subpopulations. Applied to six datasets it finds a strong correlation between age and prediction error in the Heritage Health Prize predictor, offensive labels on ImageNet for black people, and state-level pricing disparities in a Staples simulation.
AI-written summary · about
chapters (9)
- 0:00Intro
- 1:05"Unfair" associations + consequences
- 2:53Limits of preventative measures
- 5:50FairTest: a testing suite for data-driven
- 8:41A data-driven approach
- 11:16Association Guided Decision Trees
- 16:23Example: healthcare application
- 18:59Debugging with Fair Test
- 23:16Closing remarks
related talks
53mResponsible AI with SciPy
23mRemoving Unfair Bias in the Machine Learning Pipeline
42mFairness in Machine Learning: From Theory to Practice
14mBuilding with the Responsible AI toolkit
42mFairness and transparency in machine learning: Tools and techniques
31mPython and the Holy Grail of Causal Inference
14mFair And Reliable Machine Learning For High-Stakes Applications:approaches Using Information Theory
01h 11mTest-Driven Data Analysis