Believe it or not: Designing a Human-AI Partnership for Mixed-Initiative Fact-Checking
summary
A web tool queries Google for articles, runs logistic regression stance and veracity models (trained on the Emergent dataset, overall stance accuracy ~70%), and shows per-article stances and source reputations plus a combined claim score. Two user studies with ~100 Mechanical Turk participants each ask judgments on five claims; humans improve when the model is correct and degrade when it is wrong. An interactive slider to adjust source reputations lets users override signals but yields no clear average benefit. The system surfaces transparency but remains fallible.
AI-written summary · about
chapters (9)
- 0:00Introduction and motivation
- 2:54Limitations of black-box fact-checking
- 4:18Demo of mixed-initiative system
- 7:48Model performance and errors
- 8:30User study results
- 11:48Impact of interactive interface
- 13:13Broader implications and future work
- 15:18Conclusion and call to action
- 16:11Q&A on uncertainty visualization
related talks
32mRoll for Initiative: how to make the world of AI a more ethical place
26mBuild AI Systems for Discernment, Not Approval
35mAI ethics for tech writers
28mEthical AI?!
46mCan you trust your AI?
01h 21mFair Futures at Work Co Creation and AI driven Solutions in Governing
38mBuilding Responsible AI in the Age of GPT
16mTrust by Design: Building Reliable AI for Sensitive Environments