Repeatable Benchmarking: An Exploration of OpenSearch Vs Elasticsearch
summary
Compares OpenSearch v2.17 and Elasticsearch v8.15 using an open, automated framework running OpenSearch Benchmark on AWS via GitHub Actions, deploying fresh EC2 instances per test and running five daily runs (first as warmup). Reports 1500x faster on date histogram for OpenSearch and better text-query latency for Elasticsearch, measures 90th percentile service times across Big5, NYC Taxis, PMC, NOAA, Vectorsearch with thousands to hundreds of thousands samples, uses p‑value and power checks, exposes configuration experiments and reproducible outliers.
AI-written summary · about
chapters (7)
- 0:00OpenSearch vs Elasticsearch benchmarking
- 1:48Automated repeatable benchmarking
- 3:35Benchmarking methodology and setup
- 6:03Live benchmarking demo
- 8:54Recent performance improvements
- 10:20Future search engine support
- 11:37Q&A
related talks
49mInside the Elastic: Testing a Well Known Open Source Stack
06mIGNITE: How to benchmark … poorly
44mBenchmark testing
46mStop Guessing and Start Measuring - Benchmarking in Practice
29mJava Performance Testing for Everyone
44mPerformance Testing with a "Big Data Tool"
24mBetter Testing Through Statistics
43mStop Guessing and Start Measuring - Benchmarking in Practice