Crafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
summary
Step-by-step description of a Java RAG prototype that actually runs: uses LangChain for Java, Spring Boot and Angular to ingest web pages and REST JSON, splits text (visual debugger shows where split breaks answers), embeds chunks with multiple embedding models, implements a homegrown BM25 reranker to reduce context, stores vectors in PGVector and hits the PGVector 2,000-dim limit with some embeddings, compares 30+ local and cloud LLMs, measures accuracy and cost, publishes code and a VS Code/IntelliJ plugin.
AI-written summary · about
chapters (18)
- 0:00Custom RAG systems intro
- 2:48Debugging chatbot failure
- 5:14RAG pipeline breakdown
- 10:09Query routing and web search
- 11:11Custom BM25 ranker
- 13:18Chat memory and token limits
- 15:06Testing LLM providers
- 16:11Ingesting JSON/REST data
- 17:58Text splitting strategies
- 23:36Splitting techniques
- 25:22Live demo prototype
- 26:25Embedding model selection
- 31:21Vector DB selection
- 33:08Embedding model internals
- 39:11RAG performance evaluation
- 41:56Key project lessons
- 42:41IntelliJ plugin for LLMs
- 45:10Mixing LLMs in practice
related talks
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
42mTo RAG or not to RAG
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
31mEasy RAG with LangChain4J and Docker
02h 47mFrom naive to advanced RAG: the complete guide
28mNavigating Challenges and Enhancing Performance of LLM based Applications
42mTake Your RAG To The Max!
48mGoing Beyond Prompt Engineering