Faster retrieval of top N documents in Apache Solr 8.6
summary
Lucene 8.0 adds BlockMax-WAND and Solr 8.6 can use block-max indexing (requires an index built with Solr 8+) to skip non‑competitive docs and speed top‑N retrieval; enable it per query with minExactCount (opt‑in) and Solr falls back when facets or external signals are used. Benchmarks show substantial throughput gains; tradeoffs are approximate hit counts (counts become lower bounds), modest index growth (experiments report ~1–15% increase), and a distributed bug fixed in 8.8.1.
AI-written summary · about
chapters (8)
- 0:00Start
- 2:27Conjunction queries and leap frog
- 5:29Disjunction queries
- 18:33Drawbacks
- 20:23Block Max - WAND
- 24:03BMW in Lucene
- 31:07Feature Fields
- 33:07References
related talks
29mSolr Sparse Faceting #bbuzz
41mSearching large data sets in (near) constant time
30mCaching for Speed: Practical Performance Optimization in OpenSearch & Lucene
39mAccelerating Aggregations with Lucene-Aware Execution in OpenSearch
21mThe Lucene Scalpel: Tuning Low-Level Segments for Multi-Lingual Concurrency
30mBoosting OpenSearch Performance: Lucene Bulk Collection and gRPC Search in Action
41mScaling Search and Powering Vector-Based Retrieval With OpenSea...
41mE-Commerce search at scale on Apache Lucene