Search Complete
summary
Practical pipeline from raw data to hybrid search using the Amazon Product Q&A dataset: load JSON into an OpenSearch index via the bulk API, create chunks and 768-dimension KNNS vectors with all-mpnet-base-v2, store vectors with comp32x compression, and run BM25 + dense-vector hybrid queries combined with reciprocal rank fusion (default 0.5/0.5 weighting). Capacity rules: plan on ~1:1.25 active shard:CPU and target ~67% CPU, disk ≈ source size ×1.1 ×(1 + replicas) +15% overhead. Demo runs end-to-end in ~10 minutes.
AI-written summary · about
chapters (15)
- 0:00OpenSearch introduction
- 1:30Why databases need search
- 3:38Designing OpenSearch indexes
- 5:47Deploying OpenSearch clusters
- 10:22Capacity planning for nodes
- 12:29Creating indexes with mappings
- 14:14Loading data with bulk API
- 14:57Lexical search in OpenSearch
- 17:03Vector embeddings for queries
- 18:52Query types and logic
- 20:18Lexical and semantic search
- 21:00Hybrid search pipelines
- 24:58Demo: OpenSearch Launchpad
- 33:22OpenSearch in AI agents
- 35:05Summary and resources
related talks
41mText to Context: How We Introduced Hybrid Search
01h 09mVector Search: Ask Me Anything!
26mAdvancing Multi-Modal Search Capabilities in Search Pipeline
46mNatural Language & Search, Improve your Search w/ Vector capabilities
32mBudget Friendly Semantic Search With Neural Sparse Search
31mIntelligent Japanese Search With OpenSearch
32mRetrieval Augmentation and Semantic Search at Scale
30mSearch on Steroids — A Story of Semantics