Practical Strategies for Implementing Domain-Specific LLMs
summary
Claims concrete wins: a deployed RAG based inference engine on AWS that scales behind firewalls, semantic chunking outperforms fixed chunking, and BM25/DFIDF retrievers give way to POCs with dense retrievers. Evaluation uses F1 for retrieval and faithfulness and answer relevancy for generation, moving from eyeballing to supervised datasets and starting to use LLMs as judge. Retrieval stack includes vector search, a coare ranker (cross encoder), and experiments with CRAg corrective rag for confidence scoring.
AI-written summary · about
chapters (9)
- 0:00Introduction and Speaker Background
- 1:43App Architecture and Key Components
- 2:35RAG Based Inference Engine
- 4:36Evaluation and Metrics Driven Development
- 9:29Data Preprocessing Techniques
- 11:31Query Enhancement and Routing
- 14:56Retriever and Re-ranking Techniques
- 19:13Modular RAG Architecture and Future Steps
- 21:08Conclusion and Q&A
related talks
28mNavigating Challenges and Enhancing Performance of LLM based Applications
31mBeyond LLMs and Maturing RAG use cases
24mConversational Search with LLMs
57mLLMs gone wild
33mDesigning LLM Applications for Massive Datasets
53mFrom RAG to Riches: Secrets of Data-Augmented LLMs
01h 28mFrom RAG to Riches Building Enterprise Search Systems with LLMs
01h 34mHow to Construct and Assess Your First LLM Based RAG System