Bringing Down the Cost of Your OpenSearch Clusters
summary
Monitors show CPU, JVM GC time, disk skew and throughput throttle as the top cost drivers; fix unequal node load by correcting sharding and replication. Recommendations: 10–30 GB shards for search, 30–50 GB for logs, target shard sizes with ISM rollover, and enforce even distribution so hot nodes drop. Memory rules: avoid >32 GB JVM heap (keep ~50/50 heap/off‑heap) and follow the 1.5 vCPU per active shard guidance. Solutions proven in customers: dynamic sharding, snapshot‑reindex to compact mappings, and neural sparse search for low‑cost semantic queries.
AI-written summary · about
chapters (17)
- 0:00Cost optimization philosophy
- 2:28Presenters
- 3:31Key metrics for hotspots
- 8:03Cluster performance issues
- 9:29Sharding best practices
- 13:22Multi-tenancy in OpenSearch
- 15:25Scaling decisions
- 21:27Throttling issues
- 23:33QPS via search latency
- 25:40Mapping optimization
- 29:57Performance tuning
- 32:50Query writing techniques
- 34:36Cache efficiency
- 37:05Rollover index efficiency
- 39:14Slow queries with logs
- 42:41New features and tips
- 43:45Conclusion and Q&A
related talks
29mThe Search for Speed
31mMaximize Resource Efficiency with Separated Index and Search Workloads
43mMulti-tenancy for all workloads
28mRunning OpenSearch at Scale in High-Traffic Gaming Systems
40mMaking your elastic cluster perform
16mTop Elasticsearch tips
24mFinOps for Observability: Optimizing OpenSearch Costs Without Sacrificing Visibility
37mBeyond Hot Spots: Revolutionising OpenSearch Performance Through Heat-Aware Operations