How To Measure And Mitigate Unfair Bias in Machine Learning Models
summary
Hands-on workshop that builds a biased hiring pipeline and then fixes it: generates 400 synthetic CVs with OpenAI, injects names and clues, trains sklearn / LightGBM classifiers, and audits with Fairlearn and Aequitas and the Equitas toolkit. The simulated recruiter selects men at 99% vs 30% for women in examples, yielding an unmitigated equalized‑odds gap ~0.66 and balanced accuracy ~77.8%. Post‑processing threshold adjustment drops gap to ~0.47 and keeps accuracy ~78%, while a training‑time reduction gives gap ~0.48 with accuracy ~73%.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 10:09Fairness testing lab setup
- 23:29Bias in AI hiring history
- 33:41Measuring fairness in ML
- 45:58Mitigating bias with Fairlearn
- 1:00:10Fairness-accuracy trade-offs
- 1:10:29Bias in generative AI hiring
- 1:20:06Regulation and closing thoughts
related talks
01h 30mFairness Tales: How To Measure And Mitigate Unfair Bias in Machine Learning Models
23mRemoving Unfair Bias in the Machine Learning Pipeline
37mKeynote: Ethical Machine Learning: Creating Fair Models in an Unjust World
47mEthical Machine Learning
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
46mA look at the methods to detect and try to remove bias in ML models
27mCrouching Harm, Hidden Bias—Addressing issues of fairness....
51mDigital Discrimination: Cognitive Bias in Machine Learning