Evolution of Yelp search to a ranking platform
summary
Yelp migrates a custom distributed Lucene system to Elasticsearch to decouple infrastructure from relevance, moving from heap-stored maps to document_values and cutting operational overhead while serving billions of queries over 184 million reviews and ~100 million monthly uniques. Engineers implement a shading bridge to retain old analyzers, port ranking into Elasticsearch script and search plugins, and store CTR blobs in a custom doc values format. Hosting models uses the Learning to Rank plugin, which runs production rescoring and reduces cluster management pain.
AI-written summary · about
chapters (10)
- 0:00Introduction and Yelp overview
- 1:31Legacy Lucene-based search architecture
- 4:40Problems with the old system
- 7:52Requirements for new system
- 10:40Elasticsearch migration approach
- 13:30Handling Lucene version differences
- 14:56Benefits of Elasticsearch migration
- 16:44New functionality with Elasticsearch
- 18:51Learning to Rank plugin integration
- 19:36Q&A session
related talks
01h 00mElasticsearch: you know, for more than search
41mPainless / Painful? A Custom Ranking System in Elasticsearch
36mRecommendations at scale with Elastic Search
26mScaling LinkedIn’s search infrastructure
44mFinding relevancy: Improving your application search bar!
54mAdvanced (elastic)search for your legacy application
38mScaling an online search engine to thousands of physical stores
26mFrom Spark to Elasticsearch and Back