Questions and Practices to Make Algorithmic Decision Making More Fair
summary
Gives a measurement framework borrowed from social sciences and a concrete checklist of questions to ask about what is actually being measured, how proxies like income or gender are operationalized, and what audits to run. Shows failure cases: COMPAS recidivism parity vs predictive parity, Amazon hiring scrapped, image gender detectors that fail darker skin, and a teacher-evaluation that flips 6 vs 96 score instability. Explains when to collect sensitive attributes, what tests to run, and which fixes (data, labeling, deployment) actually change outcomes.
AI-written summary · about
chapters (14)
- 0:00Fairness in algorithmic decisions
- 2:31Fairness as socio-technical challenge
- 3:12Harms in automated decisions
- 7:31Affected stakeholders and groups
- 9:22Q&A on allocation bias
- 12:58Measurement models and proxies
- 15:49Real-world measurement models
- 18:18Testing construct reliability
- 21:26Evaluating construct validity
- 27:33Conversion and discriminant validity
- 30:03Consequential validity and impacts
- 31:24Fairness as contested construct
- 34:21Fairness goals and legal risks
- 35:46Q&A on fairness trade-offs
related talks
42mBe good (and don't be evil): how to audit your work for fairness and inclusion
27mCrouching Harm, Hidden Bias—Addressing issues of fairness....
35mOpen Source Fairness
28mMeasuring Model Fairness
41mMeasures and Mismeasures of Algorithmic Fairness
46mThe Measure and Mismeasure of Fairness
51mDigital Discrimination: Cognitive Bias in Machine Learning
42mFairness in Machine Learning: From Theory to Practice