Truth Checker: Generative Large Language Models and Hallucinations
summary
Claim: a post‑processing system called Truth Checker ties LLM outputs to an enterprise knowledge base and drops hallucinations to under 1%. It pairs a one‑to‑two order smaller model Elmar (vs GPT‑3’s 175B) with pre/post processors, embeddings and retrieval, fine‑tuning via LoRA/DeepSpeed, and on‑prem deployment to cut inference cost and leakage. Demo numbers: hallucination rates fall from ~14–18% to <1% and 3,000‑token API calls previously cost ≈$0.50.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:15Why is Language Understanding hard for Machines?
- 3:58Even for humans language undertaking is one of the hardest tasks
- 5:50But ChatGPT can do these tasks. How is it trained?
- 8:27What’s Happened in the last - 5 Months
- 10:45Hallucinations, When LLS make stuff up
- 13:50If not ChatGPT, which LLMs can I use?
- 20:37Running LLMs Internally can be costly
- 25:04What about Halluncations?
- 27:08ELMAR Response Generation and Truth Checker Performance
- 34:00Q&A
related talks
29mLies, damned lies and large language models
32mWhy do LLMs hallucinate?
26mReducing Hallucinations and Evaluating LLMs for Production
15mReducing Hallucinations in Conversational Systems
01h 06mWhen calibration goes awry: hallucination in language models
39mLLMs and Hallucinations: Understand, Measure, Control
10mHallucinations: Harvesting Uncertainty
35mChatGPT is lying, how can we fix it?