The Uncanny Valley of ML
summary
Argument: adding ML recommendations to human decision workflows can lower overall accuracy and slow decisions as model accuracy approaches humans, an uncanny valley of ML. Evidence: a simulated study uses perturbed labels on 100 images with a 94% human accuracy baseline via Mechanical Turk and finds system accuracy falls and decision time rises. In courts a simulated COMPAS-style case and a 400+ participant survey show ML high-risk labels drive a 14% increase in bail denial. Mitigations: simulators, clearer UI, and guardrail metrics.
AI-written summary · about
chapters (9)
- 0:00The uncanny valley of ML
- 1:52When integration goes wrong
- 4:04Testing for the valley
- 7:24ML in the judicial system
- 10:55Impact of bias and context
- 14:46Designing better systems
- 17:04Understanding the phenomenon
- 18:58Best practices for mitigation
- 22:37Call to action
related talks
49mAn AI with an Agenda
43mUX and the Changing Relationship Between Mind and Machine
41mGhost in the Machine: The Unexpected Results of Not Understanding Your Users
09mML and AI systems will make mistakes
31mThe Dark Side of AI
44mMachine Learning + Human
01h 05mNavigating the Security and Privacy Landscape of Modern AI
30mHow to Lie with AI: Understanding Bias, Ethics, and the Hidden Risks in ML