Optimizing RAG Systems in Practice: Challenges and Proven Solutions
summary
RAG pipeline diagnosis maps failures to concrete fixes: implement agentic RAG with web search tool that makes function calls to fetch missing facts, compare chunk sizes (500–1000) and tune top‑k/similarity thresholds (e.g. require >60%), replace out‑of‑date vectors via the MTB embedding leaderboard to pick newer models, add a ranker and context‑compression step, and prefer semantic chunking or hybrid keyword+semantic retrieval. Long‑context LLMs beat RAG on accuracy but spike token costs; a selective hybrid (route hard queries to long context) cuts cost while reducing hallucination.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:14RAG systems overview
- 2:41RAG challenges
- 4:07Missing content in RAG
- 8:17Miss-ranked retrievals
- 10:49Better retrieval strategies
- 15:01Privacy and embeddings
- 17:27Advanced retrieval techniques
- 18:29Context compression
- 19:33Fixing response formats
- 21:41Incomplete responses
- 22:44Query rewriting for retrieval
- 23:48RAG vs long-context LLMs
- 25:56Key takeaways and future of RAG systems
related talks
19mImproving RAG Systems - Key Challenges and Solutions
15mEvaluating RAG Model Performance Metrics, Bias, and Interpretability
59mThe Evolution of RAG
19mRAG in 2025: State of the Art and the Road Forward
56mRAG: from dumb implementation to serious results
28mOptimizing Context Tuning for RAG Systems
42mTo RAG or not to RAG
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance