Introducing OpenSearch’s Search Relevance Workbench!
summary
Search Relevance Workbench shows how to measure and improve search quality end to end: it introduces Search Configurations as named, versioned query+index+pipeline bundles, offers eyeballable side‑by‑side result diffs and query‑set comparison with Jaccard, RBO and NDCG, and runs evaluations with judgments from uploads, implicit clicks via User Behavior Insights or LLM as a Judge. It supports scheduled nightly checks, interleaved A/B testing, APIs to compare other engines, and a Hybrid Optimizer for automated parameter search and per‑query tuning.
AI-written summary · about
chapters (18)
- 0:00OpenSearch Relevance Workbench intro
- 1:03Core of search relevance
- 3:09Measuring user behavior
- 4:13What the Search Relevance Workbench offers
- 5:55Use your own data
- 7:40Eyeballing tools
- 9:08Live demo
- 13:27Query sets
- 18:26Judgments evaluation
- 20:51Classic metrics
- 25:08Three workbench steps
- 26:32Search configurations
- 27:58Future features
- 30:27Other platforms evaluation
- 31:52Solr to OpenSearch
- 33:43Next steps resources
- 35:07Agentic relevance tuning
- 36:52Q&A
related talks
40mAgentic Relevance Tuning With OpenSearch: Faster, Easier, and Higher Quality Search at Scale
31mSponsored Session: From Clicks to Rankings: Building a Learning-to-Rank Pipeline with…
40mLevel Up: Everything you need to know about search, ...
37mSearch and Recommendations: 3 Sides of the Same Coin #bbuzz
31mIntelligent Japanese Search With OpenSearch
26mTeaching Ruby to Rank: A Search Pipeline Story
42mContribute to OpenSearch: Build Your Skills While Advancing Open Technolo...
38mLearning to Rank: where search meets machine learning #bbuzz