Scaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...
summary
Uber runs OpenSearch across 100+ clusters with typical clusters at 4–5 TB and 150+ nodes, using containerized OpenSearch plus sidecars and a goal‑state worker → manager → controller pipeline for automated, sema‑controlled rolling upgrades that achieve zero‑downtime. Active‑active regional deployment (DCA/PHX), shard allocation awareness with three isolation groups limits data loss to ~33%, 50–60 alerts per production cluster, moving‑window leader/follower monitoring, periodic zonal resiliency drills, and combined vertical/horizontal autoscaling deliver about $1M/year savings.
AI-written summary · about
chapters (13)
- 0:00Scaling OpenSearch at Uber
- 1:04Search infrastructure evolution
- 8:26OpenSearch scale and use cases
- 10:00Resilient infrastructure design
- 14:51Cluster deployment automation
- 20:26Monitoring and recovery automation
- 22:32Zero-downtime upgrades
- 26:26Release pipeline for clusters
- 29:37Zone failure handling
- 34:18Resiliency drills
- 35:01Scaling strategies
- 39:16Multi-zone operational challenges
- 42:06Q&A
related talks
43mMulti-tenancy for all workloads
43mDeep Dive Into OpenSearch Pull-Based Ingestion at Uber
28mRunning OpenSearch at Scale in High-Traffic Gaming Systems
38mMigrating from Self-Managed ElasticSearch to OpenSearch
38mEvolution of Uber's Search Platform
31mMigrating from Self managed Elasticsearch to OpenSearch & How Indias Fantasy Sports Giant Leveled Up
31mMaximize Resource Efficiency with Separated Index and Search Workloads
31mSharding 2.0: Unlocking Infinite Search Scalability in OpenSearch