Enhancing AI and LLM Systems with Hybrid Search
summary
Hybrid search combines keyword filters with vector retrieval to fix real errors: using k-NN and HNSW for ANN, then applying metadata pre‑filters stopped Kung Fu Panda results from returning Enter the Dragon; an e‑commerce flow narrows billions to a candidate set and rescoring uses TF-IDF/BM25 rarity boosts plus weighted score or rank‑fusion to merge lexical and semantic signals. A startup called hyperspace ships a real‑time hybrid search DB and a virtual chip approach to cut latency and cost at billion‑document scale.
AI-written summary · about
chapters (10)
- 0:00Hybrid search for AI systems
- 3:15Vector search limitations
- 7:10Semantic meaning in vectors
- 12:33Vector search challenges
- 14:43Combining vectors and filters
- 20:29Rarity scoring in classic search
- 24:50Merging search scores
- 28:06Scaling to billion documents
- 30:59Virtual chip for search
- 31:41Q&A
related talks
49mHybrid search on hybrid models, at scale
22mAgentic Hybrid Retrieval: Orchestrating LLM Agents, Hybrid Search & Explainability in OpenSearch
45mRevolutionizing Search with Power of Neural Sparse and Dense Vectors
30mBeyond Keywords: Building an AI-Powered Search That Reads Your Mind
41mResearch in the Age of AI
47mBoosting AI With OpenSearch: Vector Stores, Similarity Search, and Retrieval-Augme...
09mUnconference: How will AI + Search (IR) converge?
49mLLM—Search Engine Killer?