Flagging *and* Fixing Bias in ML
Talkat Open AI + Data Forum @ Open Source Summit North America 2022 · on Thursday, 2022-06-23· at 16:05· Room 303/304 (Level 3), JW Marriott Austin, 110 E 2nd Street Austin, TX 78701·
summary
A practical roadmap that flags where bias enters — problem framing, data, training, evaluation, deployment — and offers concrete fixes: Know Your Data finds proxies and sample skews; apply upsampling or synthetic data, use the model remediation toolkit (mendef loss) or TensorFlow Constrained Optimization to enforce demographic parity or equality of opportunity, and use counterfactual logit pairing to close toxicity gaps. Tools for sliced metrics include Fairness Indicators, What‑If and the Language Interpretability Tool, and Model Card Toolkit documents limitations. The talk cites the shoe‑drawing game, a Boston study riddle, and mentions 21 fairness definitions and 101 crash deaths as cautionary examples.
AI-written summary · about
chapters (18)
- 0:00Start
- 2:44What Is Machine Learning
- 3:30Interaction Bias
- 3:57Selection Bias
- 5:08Typical Ml Workflow
- 5:42Automation Bias
- 6:38Sampling Bias
- 8:04Deployment Bias
- 10:36Defining a Problem
- 12:33Definitions of Fairness
- 13:25Demographic Parity
- 14:16Equality of Opportunity
- 15:20Collecting and Preparing Data
- 17:30Tainted Samples and Proxies
- 23:10Fairness Indicators
- 23:46Confidence Intervals
- 25:34Model Card Toolkit
- 28:45The Open Images Challenge
related talks
27mCrouching Harm, Hidden Bias—Addressing issues of fairness....
47mFight AI Bias With… Bias
38mSocial Implications of Bias in Machine Learning
47mEthical Machine Learning
51mDigital Discrimination: Cognitive Bias in Machine Learning
42mFairness in Machine Learning: From Theory to Practice
46mA look at the methods to detect and try to remove bias in ML models
38mBiases are bugs: algorithm fairness and machine learning ethics