Disincentivizing Hallucination
summary
Joint work with Santosh Vempala, Ofir Nachum and Edwin Zhang argues hallucinations arise because benchmarks reward guessing, not humility, and demonstrates concrete fixes. A three-call "humble" reduction (ask twice, reconcile with a third) cuts false outputs dramatically but also lowers accuracy unless the scorer changes. On a SimpleQA evaluation ChatGPT, Claude, Gemini and Grok show this tradeoff; Gemini is most accurate yet rarely says “I don’t know.” An open rubric with 50% credit for “I don’t know” removes the tradeoff and incentives adoption.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:03Defining hallucinations
- 3:47Hallucinations as guessing
- 7:05Humble reductions
- 12:41Open rubrics for humility
- 19:31Mathematical breakthroughs
- 24:22Misconceptions about hallucinations
- 25:45Conclusion
related talks
15mReducing Hallucinations in Conversational Systems
10mHallucinations: Harvesting Uncertainty
32mWhy do LLMs hallucinate?
06mInterview with Spela Poklukar
41mTruth Checker: Generative Large Language Models and Hallucinations
37mAI Hallucination Explained 🤯 Why AI Gets Things Wrong
47mYour Test Suite Can’t Catch a Hallucination: Talk on AI in Production
01h 06mWhen calibration goes awry: hallucination in language models