Exploring reranking depth in modern search pipelines
summary
Experiments use BM25 + Elasticsearch index and multiple cross‑encoder rerankers evaluated on BEIR and similar datasets with NDCG@10, measuring rerank depth K. Three empirical patterns emerge: 72.6% parto pattern (rapid gain then saturation), 20.2% unimodal (gain then decay) and 7.1% bad‑fit. A Pareto curve fits many curves well; oracle shows remaining headroom. Causes for unimodal include sparse relevance labels and negatives sampling bias; expanding negative mining and training depth improves robustness. Latency‑aware plots show 90% gains at roughly one‑third the max depth.
AI-written summary · about
chapters (10)
- 0:00Introduction to reranking depth
- 1:52Experimental setup overview
- 3:19Most common pattern analysis
- 10:30Unimodal pattern explanation
- 17:34Bad fit pattern causes
- 20:28Interpretability via score distribution
- 21:51Efficiency versus effectiveness trade-off
- 26:07Latency-aware model comparison
- 28:39Related work discussion
- 33:19Q&A session
related talks
29mThe Art of Production Quality Retrieval
26mTeaching Ruby to Rank: A Search Pipeline Story
33mNeural Search - Let's talk about quality
37mLexical, Vector & Hybrid Search with Elasticsearch
47mLearning to rank search results
33mAI-Powered Search in Modern E-Commerce
40mLevel Up: Everything you need to know about search, ...
25mBetter search relevance using Learning to Rank at mobile.de