What can AI do for me?: evaluating machine learning interpretations in cooperative play
summary
Evaluates whether model explanations actually improve human decisions by pairing Quizbowl players with Quanta, an elastic‑search QA model, and measuring correctness and buzzing time. System shows three interpretations — alternative answers, training-example evidence, feature highlights — across eight on/off combinations on 160 unseen questions with 30 experts and 30 novices. Results show interpretations speed decisions and help experts when combined, but novices gain more yet often over‑trust and get misled; dynamic selection and user training improve outcomes.
AI-written summary · about
chapters (10)
- 0:00Introduction to AI interpretations
- 1:53Human-centered ML evaluation
- 5:25Quizbowl task overview
- 7:42Interpretation forms in Quanta
- 9:51Experiment design and data
- 10:38Regression analysis results
- 12:45Novice vs expert interpretation effects
- 13:50Speed and accuracy trade-offs
- 14:56Key takeaways and limitations
- 16:22Q&A session begins
related talks
41mIf We Want AI to be Interpretable, We Need to Measure Interpretability
42mQA for ML: How we can trust AI with Food Sustainability
46mCan you trust your AI?
01h 05mAI: A Return to Meaning
42mAI & Human Decision Making
01h 01mIntroduction to Practical AI
30mTrustworthy AI - opening up the black box
34mInterpretable AI or How I Learned to Stop Worrying and Trust AI