Federated Vector Retrieval Across Organizations With Milvus, OPA...
Talkat Open Source Summit Japan + AI_dev: Open Source GenAI & ML Summit + Automotive Linux Summit 2025 · 2025-12-08..12-10·
summary
Architecture proposes a federated vector retriever that generates JSON summaries from metadata, embeds them into a semantic catalog, and runs a two‑stage search: candidate selection (semantic catalog) then provider‑side similarity search with the provider's embedding model, aggregating top‑K results. Implementation uses vLLM for high‑throughput summarization, Milvus as the internal catalog and scale target (thousands of sources, billion‑scale capable), OPA for millisecond policy checks and dynamic access control, and LangChain for orchestration; prototype is feasible but flagged as an early build with limited privacy‑enhancing tech.
AI-written summary · about
chapters (19)
- 0:00Federated vector retrieval intro
- 1:04RAG and cross-organizational data
- 3:12Why cross-organizational sharing matters
- 4:34Challenges of cross-organizational RAG
- 9:11Fed Vector Retriever concept
- 10:39Data source summarization
- 12:48Federated similarity search
- 14:55Policy-aware retrieval
- 16:43End-to-end workflow
- 17:47Retrieval with policy enforcement
- 19:11Open source reference
- 22:05VLM for high-throughput summarization
- 24:22Summaries with LangChain
- 26:29Vector database requirements
- 28:40Milvus for semantic catalog
- 30:08Why Open Policy Agent for authorization
- 32:59Policy decision-making with OPA
- 34:48Key takeaways
- 36:13Q&A on legal and privacy
related talks
26mThe Rise of Vector Databases - Lessons from the Milvus Community
37mMilvus: Scaling Vector Data Solutions with Gen AI
35mMilvus 2.0: Building the World's Most Advanced Open-source Vector Database
36mMilvus: Scaling Vector Data Solutions with Gen AI
41mGetting Started With Vector Databases
51mUnleashing the Potential of Cloud-Native Vector Databases
40mEvolution of Milvus Cloud-Scalable Vector Database
50mVector Ops: run vector embedding-powered apps in prod