Scaling OpenSearch: Next-Generation Shard Allo...
summary
Describes concrete changes that cut full-cluster reroute from 232 minutes to about 40 minutes on a 1,000-node test with 500k shards on 1,000 nodes: batch-fetching gateway checks, caching disk and decider state, removing needless string formatting, and making allocators time-bound reroute iterations so work yields every ~40 seconds. Shows decider tweaks (throttling, awareness, disk threshold) and reports up to 93% faster cluster-state processing and P99 reroute down to ~40s, with practical tuning knobs.
AI-written summary · about
chapters (8)
- 0:00Scaling OpenSearch shard allocation
- 1:01How shard allocation works
- 10:03Scaling challenges in large clusters
- 18:34Profiling allocation deciders
- 21:03Time-bounding allocators
- 25:56Performance after optimizations
- 33:00Future scope for shard allocation
- 35:08Q&A
related talks
31mSharding 2.0: Unlocking Infinite Search Scalability in OpenSearch
30mUnderstanding Bottlenecks & Conquering them
44mScaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...
44mOpenSearch’s Quest for Disaggregated Storage: The Strangler Fig Patter...
43mMulti-tenancy for all workloads
16mTowards Optimal Rack-scale μs-level CPU Scheduling through In-Network Workload...
31mMove Less, Move Faster: Speeding Up Citus Cluster Scaling
19mYou Know, for Search. Querying 24 Billion Records in 900ms