C++ Search for Database Kernels: Built In, Not Bolted On
summary
IResearch is an Apache 2.0 C++ library built to embed inside database kernels with WAL integration and transactional consistency and it benchmarks against Lucene and Tantivy using the search-benchmark-game. The engine uses block-at-a-time scoring with AVX2 SIMD, two block sizes (32 for conjunctions, 128 for disjunctions), and a hybrid push-down/late-scoring strategy. Results show wins in top100+count including 3.6x faster on unions. The team shares concrete mistakes, integration rewrites, and production optimizations.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:33Search market history
- 4:11Performance benchmarks
- 7:57The scoring challenge
- 11:27Workarounds and limits
- 14:02Block scoring strategy
- 16:55Conjunctions and disjunctions
- 18:51Greedy evaluation
- 20:18Top-K retrieval
- 22:06Conclusion and benchmarks
related talks
01h 16mHow do to do research with Systems Researchers
40mUsing Native CPU Vectorized Instructions for 60% Speedup on Lucene's KNN Vector Search
36mCrateDB: A Search Engine or a Database? Both! #bbuzz
26mOn The Look-Out For Your Data
43mOpenSearch and the Path to a Trillion-scale Vector Database
28mElasticsearch-Quality Full-Text Search in Postgres via Tantivy
49mBuild a database from the ground up
05mKeynote: OpenSearch: From Code to Community