The Elephant in your Dataset: Addressing Bias in Machine Learning
summary
Walks through the ProPublica COMPAS dataset (~7,214 rows), reproduces the recidivism score with a simple logistic regression (~76% accuracy) and shows Black defendants have 2x higher false positives while White defendants have ~2x higher false negatives; demonstrates that dropping race barely changes results, balancing the target ratio reduces disparity but shrinks data and hurts accuracy, and contrasts real-world failures like Google’s gorilla label and a 34.4% error gap in facial APIs; recommends pre-/in-/post-processing mitigation and governance.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 1:28Disclaimers
- 3:16Bias as socio-technical issue
- 5:02Privilege and bias exercise
- 8:59Bias in society and ML
- 12:12Necessary bias in ML
- 14:04Harm in ML lifecycle
- 16:37Seven harmful bias types
- 24:32Aggregation bias in models
- 27:09Accuracy versus fairness
- 28:55Deployment bias risks
- 30:41Bias in generative AI
- 32:52Measuring and mitigating bias
- 33:16Protected attributes limits
- 34:40Equitable AI research
- 36:30Measuring fairness with mathematical concepts
- 40:02Mitigation strategies
- 43:18COMPAS bias demo
- 52:04Socio-technical solutions
- 54:54Ethical debt in AI
- 57:49Q&A on bias
related talks
47mThe Elephant in your Dataset Addressing Bias in Machine Learning
55mEthical Machine Learning
31mDoes This Algorithm Have a Soul?
43mThe Elephant in your Dataset: Addressing Bias in Machine Learning
31mAvoiding Digital Bias
05mHow to Erase Unconscious Bias From Your AI Datasets
23mRemoving Unfair Bias in the Machine Learning Pipeline
47mEthical Machine Learning