Solr Eclipse - Zomato's Journey to 64 Million Search Queries a Day
summary
Starts from one Solr instance serving ~1k queries/day for 10k restaurants and scales to 64 Million queries/day, a catalog of 100M SKUs / 800k+ restaurants / 50k dishes across 13 languages, using EFS incremental backups and warm‑up queries to cut replica boot traffic from gigabytes to kilobytes, JVM and HTTP pool tuning, English‑only cluster splitting to shrink indexes, and a SolrCloud design with NRT, tlog and pull replicas plus 33% on‑demand / 67% spot instances to avoid leader-recovery downtime. Overall architecture succeeds at reducing boot time and preventing full-cluster outages.
AI-written summary · about
chapters (7)
- 0:00Introduction and background
- 3:21Spot instances and EFS backups
- 7:13Cluster splitting for scalability
- 8:58Master failure and recovery
- 10:40SolrCloud architecture shift
- 13:46Final SolrCloud setup and recovery
- 15:11Q&A
related talks
41mScaling Search and Powering Vector-Based Retrieval With OpenSea...
29mScaling Search Clusters with Apache Solr and Kubernetes
41mE-Commerce search at scale on Apache Lucene
41mSearching large data sets in (near) constant time
44mScaling Apache Solr -- Features Of Resilience
19mEnhance Search Performance At Scale with
28mRunning OpenSearch at Scale in High-Traffic Gaming Systems
33mLessons From Migrating To OpenSearch: Shard Design, Log Ingestion, and UI Decisions