Intelligent Japanese Search With OpenSearch
summary
Shows concrete recipes: uses Sudachi tokenizer, script normalization and synonym expansion to prepare Japanese text, indexes Amazon Reviews and runs controlled comparisons of BM25 vs dense vector (HNSW/IVF) vs sparse models and a hybrid+sparse+BM25 fusion, measuring ranking metrics and latency. Demonstrates on-disk and quantized runs that cut memory to ~1/16 at measurable accuracy loss, uses OpenSearch neural search pipeline and neural search pipeline to embed at ingest and query time, and reports tradeoffs for chunking, re-ranking, and production deployment.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 2:27Search system overview and retrieval
- 3:49Lexical search basics for Japanese
- 9:04Dense vector search mechanisms
- 12:36Neural search in OpenSearch
- 14:38Sparse vector search overview
- 17:51Hybrid search scoring methods
- 20:43Document processing and units
- 22:31Beyond retrieval search enhancements
- 23:34Reranking techniques and use
- 24:59Query expansion and agent search
- 26:26Demo cases and practical strategies
- 28:05Summary and Q&A
related talks
32mBudget Friendly Semantic Search With Neural Sparse Search
46mNatural Language & Search, Improve your Search w/ Vector capabilities
46mDelivering the next wave of innovation with OpenSearch
48mUnlocking Next-gen Semantic Search With OpenSearch 3.x
32mRetrieval Augmentation and Semantic Search at Scale
41mText to Context: How We Introduced Hybrid Search
26mAdvancing Multi-Modal Search Capabilities in Search Pipeline
36mSearch Complete