Smart search over internal knowledge base with LLMs: from architecture to deployment
summary
Describes building an intelligent internal knowledge-base search using large language models (LLMs) and retrieval-augmented generation (RAG) to speed up work for product managers, developers, and testers. Covers real usage examples, system architecture, quality evaluation, and practical issues encountered during the system's evolution and deployment.
AI-written summary · about
chapters (31)
- 0:00Introduction
- 1:09Internal search problems
- 3:57Solution architecture
- 4:59RAG pattern
- 6:47Search quality evaluation
- 8:31Synthetic benchmarks
- 13:07Chunking methods
- 17:18Structured documents
- 21:10Combining search methods
- 22:38Ranking strategies
- 25:06Domain terms handling
- 26:51Long ticket processing
- 28:17Question complexity
- 30:23Final architecture
- 31:05Key learnings
- 33:57Summary
- 34:19Q&A: Team size
- 35:45Q&A: MCP integration
- 36:27Q&A: Security & retraining
- 38:32Q&A: Dialogue context
- 39:59Q&A: Response time
- 42:27Q&A: Dataset validation
- 44:32Q&A: Parallel projects
- 45:37Q&A: Document relationships
- 46:43Q&A: Prompt tuning
- 47:27Q&A: Benchmark types
- 48:31Q&A: Dynamic sizing
- 50:59Q&A: Outdated docs
- 52:22Q&A: Chat pipeline
- 53:48Q&A: LLM necessity
- 55:38Q&A wrap-up
related talks
40mPractical Lessons from Deploying AI Assistants Based on LLMs
19mHow I Learned to Stop Worrying and Love the AI — Integrating LLMs into a DMS
49mLLM—Search Engine Killer?
51mQuery Understanding With Large Language Models: Techniques and...
24mConversational Search with LLMs
32mTruth-seeker: Using LLM agents to build and verify knowledge bases
32mBuilding Multi-Agent RAG Systems
53mFrom RAG to Riches: Secrets of Data-Augmented LLMs