Budget Friendly Semantic Search With Neural Sparse Search
summary
Walks through a working Neural Sparse Search pipeline on OpenSearch: deploys a doc-only encoder and tokenizer (config copied from docs), indexes Amazon PQA with an ingest pipeline that produces rank_features, and runs neural sparse queries whose scores are the dot product of weighted term expansions (SPLADE-style). Shows lexical BM25 vs neural sparse relevance, uses pruning (absolute or top-percentage pruning, e.g. top 40%) to cut index size and latency, recommends remote hosting (SageMaker/Bedrock), notes sparse is cheap up to ~10M and ANN raises that toward ~100M.
AI-written summary · about
chapters (17)
- 0:00Neural sparse search intro
- 1:23Vocabulary mismatch problem
- 3:13SPLADE for term expansion
- 6:25Doc-only vs bi-encoder models
- 7:54Demo setup
- 8:41Deploying doc-only encoder
- 11:04Ingestion pipeline for vectors
- 13:34Indexing with sparse expansion
- 15:18Deploying tokenizer model
- 15:38Running search with scoring
- 19:10Neural vs lexical search
- 21:38Limitations and hybrid search
- 22:21Pruning for speed
- 25:14Simplified sparse search
- 26:19Why sparse vectors are budget-friendly
- 28:50Best practices for implementation
- 31:02OpenSearch best practices
related talks
13mFast and Explainable Search on a Budget With OpenSearch Neural Sparse
46mNatural Language & Search, Improve your Search w/ Vector capabilities
31mIntelligent Japanese Search With OpenSearch
41mFine-tuning Neural Sparse Model for Domain Specific Data From an Existing OpenSearch Index
30mSemantic Search and Neural Information Retrieval
48mUnlocking Next-gen Semantic Search With OpenSearch 3.x
47mWhy, What, When and How of Running Diff Types of Sear...
13mA Beginner’s Guide to Semantic Search