First-Person Fairness in Chatbots
summary
Defines first-person fairness as fairness toward the individual user and gives a repeatable pipeline: use public chat corpora (LMSYS, WildChat), a language-model grader to label millions of chats, and crowdsource checks (LM–human correlation ≈ 0.86). Measured harmful stereotypes and quality gaps by replaying prompts with different names as demographic proxies; earlier pre‑RL models show biases up to ~5% in some tasks (stories highest), while post‑training reinforcement learning cuts those biases to typically under 1%, demonstrating a scalable, effective mitigation.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:07Fairness in chatbots matters
- 9:15Evaluating chatbot fairness
- 15:21First-person fairness definition
- 23:13Gender bias examples
- 30:15Post-training bias reduction
- 35:13Measuring harmful stereotypes
- 39:31Response differences across demographics
- 40:15Conclusion and future
related talks
40mResponsible AI Practices: Fairness in ML
01h 30mFairness Tales: How To Measure And Mitigate Unfair Bias in Machine Learning Models
20mLet’s Talk About Race: Identity, Chatbots, and AI
12mSafety Perceptions of Generative AI Conversational Agents: Uncovering Perceptual...
10mPrivacy and Respectful Discourse in AI Chatbots
01h 21mFair Futures at Work Co Creation and AI driven Solutions in Governing
01h 05mIs ChatGPT (and Other AI) the Enemy of Diversity?
09mBuilding Trust in Conversational AI