The Power of RAG at Scale: A Year of Explosive Growth and Observab...
summary
Claims that Retrieval-Augmented Generation reduces hallucinations but still fails when retrieval returns mixed records are illustrated with a concrete error: retrieval merged contacts for "Shyak" and "Shamik" and produced a false attribution. The pipeline uses TruLens open-source for logging, feedback functions and evals, Docker/Kubernetes (minikube, YAML, HPA), vector DBs like Pinecone, and custom smaller eval models to cut cost versus OpenAI. Evaluation focuses on the RAG triad — context relevance, answer relevance and groundedness score and notes ~80% LLM-human agreement from related research.
AI-written summary · about
chapters (11)
- 0:00RAG at scale intro
- 1:04Hallucinations in RAG
- 2:09True Lens for evaluation
- 4:17Observability platform needs
- 5:21Why scaling observability matters
- 8:10Evaluating LLM apps at scale
- 10:17True Lens in containers
- 13:55Scaling True Lens with K8s
- 15:18Optimizing evaluation
- 16:44Open source & CNCF
- 17:27Q&A
related talks
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
42mTo RAG or not to RAG
53mFrom RAG to Riches: Secrets of Data-Augmented LLMs
14mBeyond RAG: Building Reliable AI Systems for Privacy Assessments
31mAdvanced Enterprise RAG Systems
39mRAGs to Riches: How Our Content Affects Retrieval Augmented Generation
50mRetrieval Augmented Generation Beyond Toy Demos
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production