LLMs and Hallucinations: Understand, Measure, Control
summary
The talk examines causes, detection, measurement and mitigation of hallucinations in large language models (LLMs), focusing on their impact in high-stakes domains like healthcare and finance. It covers detection and evaluation methods, reduction strategies, and context-engineering techniques such as agentic workflows and retrieval-augmented generation (RAG), plus architectural choices that improve reliability for specific use cases.
AI-written summary · about
chapters (25)
- 0:00LLMs and hallucinations intro
- 1:02Hallucination definition
- 3:49Main sources
- 5:13No Free Lunch theorem
- 6:37Preventing factual hallucinations
- 8:00Complacency bias solutions
- 10:29Hallucination robustness pyramid
- 11:32Context engineering
- 12:35Post-generation quality control
- 14:02Agentic orchestration
- 15:04Hallucination detection techniques
- 16:34Probing presentation
- 19:03Building robust AI systems
- 20:26Model selection benchmarks
- 21:53RAG for product info
- 23:18Preventing calculation errors
- 23:59Response validation
- 25:02Residual hallucination management
- 26:06Evaluating hallucinations
- 27:55Factuality benchmarks
- 30:23Fidelity evaluation
- 31:45Practical use-case evaluation
- 32:49Guardrails evaluation
- 33:53Model comparison resources
- 34:23Conclusion
related talks
32mWhy do LLMs hallucinate?
26mReducing Hallucinations and Evaluating LLMs for Production
49mA Practical Approach to Using LLMs: Peculiarities and Pitfalls
39mHallucination-Free LLMs: Strategies for Monitoring and Mitigation
29mWhat Is It Like to Be an LLM?
37mAI Hallucination Explained 🤯 Why AI Gets Things Wrong
07mLightning Talk: Grounding LLM to Avoid Hallucinations Using Vector S...
28mWhat Are AI Hallucinations?