Building Privacy-First Search With Containerized OpenSearch and Local LLMs
summary
A privacy-first RAG pipeline runs entirely on-prem using containerized OpenSearch (KNN/HNSW vectors, hybrid BM25 support) plus local LLMs served from LM Studio (nomic-embed text v1.5 for embeddings, Qwen 3 / Qwen-34B for inference), deployed via Docker and a simple Streamlit UI. The demo ingests PDFs, indexes vectors (index params ~100, custom mappings), and answers queries with retrieved context and citations; claim supports 100K+ token contexts and the local demo returns correct, cited responses.
AI-written summary · about
chapters (11)
- 0:00Privacy-first search intro
- 1:07Avoid proprietary AI APIs
- 2:32OpenSearch for vector search
- 3:38System architecture
- 5:03Local OpenSearch setup
- 6:51Demo begins
- 13:26RAG pipeline interaction
- 15:39Repository and resources
- 16:20Adding MCP context
- 17:26Scaling clusters
- 18:16Q&A
related talks
21mBuilding RAG With OpenSearch ML Plugins: From PDFs To Voice-Enabled Search
26mOpenSearch: Conversations with Context
49mLLM—Search Engine Killer?
34mImplementing an open-source RAG with OpenSearch
51mQuery Understanding With Large Language Models: Techniques and...
38mAI-Ready Data Foundation Using Amazon OpenSearch
17mFine-Grained Authorization for Secure RAG With OpenSearch
22mAgentic Hybrid Retrieval: Orchestrating LLM Agents, Hybrid Search & Explainability in OpenSearch