The Dos & Don'ts of ElasticSearch Scalability & Performance
summary
ElasticSearch mappings must be chosen up front: dynamic mapping explodes into a 40 MB mapping at one site, and changing analyzers forces full reindex. Over-sharding creates query blowups — a cluster with ~1000 shards turns each search into thousands of subqueries. Use routing (user ID) to localize data, prefer dedicated master nodes, limit fielddata (recommend ~40% heap) to avoid OOMs, and tune refresh (default 1s can be raised to 30s). Measure in tests, monitor GC/thread pools, and plan zero-downtime reindexing.
AI-written summary · about
chapters (16)
- 0:00Introduction to ElasticSearch
- 1:37Mapping and Schema Design
- 8:28Data Modification Options
- 10:58Shard Distribution Basics
- 14:03Index Partitioning Strategies
- 18:22When to Avoid Extra Shards
- 20:15Node Roles and Cluster Stability
- 22:41Client Types and Performance
- 24:11Query Optimization Techniques
- 26:27Efficient Query Engineering
- 28:05Filtering and Performance
- 30:02Understanding Field Data
- 33:06Reindexing Considerations
- 34:34Production Setup Best Practices
- 38:05Production Environment Testing
- 39:33Final Recommendations and Resources
related talks
04mOpenSearch Performance and Monitoring
33mLessons From Migrating To OpenSearch: Shard Design, Log Ingestion, and UI Decisions
30mCaching for Speed: Practical Performance Optimization in OpenSearch & Lucene
31mMaximize Resource Efficiency with Separated Index and Search Workloads
41mScaling Search and Powering Vector-Based Retrieval With OpenSea...
28mRunning OpenSearch at Scale in High-Traffic Gaming Systems
19mEnhance Search Performance At Scale with
29mScaling Search Clusters with Apache Solr and Kubernetes