Lessons learned in bringing a RAG chatbot with access to 50k+ diverse documents to production
summary
Production RAG system at Merck Electronics indexes 50,000+ documents from SharePoint and experiment writeup DBs, built on Palantir Foundry with PySpark/Polars, Quadrant vector store, OpenAI dense embeddings plus BM25 sparse for hybrid search, Streamlit UI on AWS Fargate, Langfuse tracing and permission-aware filters. CI runs ~130 PyTest cases including keyphrase checks and LLM-as-judge evaluations. Outcome: ~150 users, ~80k documents live, higher test pass rates after adding hybrid search, but months of engineering with 1–3 devs.
AI-written summary · about
chapters (8)
- 0:00Introduction and agenda
- 1:28Use case at Merck
- 4:26Data ingestion pipeline
- 13:08Chatbot workflow
- 22:09Evaluation framework
- 29:20Development workflow
- 34:06Lessons and takeaways
- 39:18Q&A session
related talks
01h 36mTutorial: From Planni...
41mFrom RAGs to riches: Build an AI document interrogation app in 30 mins
48mUnlocking LLM Potential: From Exploration to Integration
36mDon't tell me RAG is easy
42mBuilding Production RAG Systems for Health Care Domains
23mNo RAGrets
01h 28mFrom RAG to Riches Building Enterprise Search Systems with LLMs
42mTake Your RAG To The Max!