Measures and Mismeasures of Algorithmic Fairness
summary
Concrete failures and fixes instead of abstraction are the deciding factors here: Google Cloud Vision mislabels gender and Google Photos once tags Black people as gorillas, an error tied to training sets; commercial face systems show much higher error rates on dark‑skinned women in Gender Shades; Amazon abandons an AI recruiting system after it downgrades women; COMPAS gives higher false positives for Black defendants; proposed fixes include data sheets for datasets and model cards to document provenance, intended use, and subgroup performance.
AI-written summary · about
chapters (15)
- 0:00Algorithmic fairness intro
- 1:32What is fairness and why it matters
- 3:17Critique of fairness math
- 6:04Legal and societal fairness
- 7:50Bias types in practice
- 14:03Measuring fairness mathematically
- 17:12Fairness models and adversarial training
- 21:22Limitations of anti-classification
- 23:47Classification parity and demographics
- 28:42False positives and equal opportunity
- 30:47Calibration and risk fairness
- 33:54Fairness metrics incompatibility
- 35:18Dataset and model documentation
- 38:02Ethical AI organizations
- 40:09Conclusion
related talks
37mThe AI 'Black Box'
47mEthics & AI: Identifying & preventing bias in predictive models
38mSocial Implications of Bias in Machine Learning
27mCrouching Harm, Hidden Bias—Addressing issues of fairness....
47mDiversity and Inclusion for Algorithmic Bias Considerations
21mFairness and bias in AI
47mEthical Machine Learning
41mWhen AI Failures Reflect Our Own Social Biases