Introduction to Search
summary
WordPress.com builds a reader search on Elasticsearch that indexes content from 90 million posts per month, serving about 60 million queries/day and ~12 million index ops/day. The pipeline shows concrete choices: tokenizers and light stemming for queries as-you-type, per-field analyzers, a query builder that composes must/should clauses and aggregations, and rankers using TF‑IDF/BM25 (not naively) with link signals like PageRank. Quality metrics are pragmatic — interaction with the first ten results drives A/B tests and iterations.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:06Our company and data
- 3:42User search intent types
- 5:56User expectations from search
- 9:10Core search engine components
- 10:33How indexing works
- 12:25Text tokenization challenges
- 14:33Text analyzers compared
- 16:00Stemming vs lemmatization
- 17:27Elasticsearch for indexing
- 18:33Query builder basics
- 21:02Writing Elasticsearch queries
- 22:30Advanced Elasticsearch queries
- 24:16Query builder advanced
- 25:22Ranking algorithms intro
- 26:45TF-IDF limitations
- 27:49Improving ranking with BM25
- 29:16PageRank and link analysis
- 31:50Search presentation importance
- 33:16Resources and closing
related talks
40mLevel Up: Everything you need to know about search, ...
01h 30mBuilding a Search Engine
42mBuilding a real-time news search engine #bbuzz
26mFrom __icontains to search
16mWhy we built ElasticSearch
24mSearch From the Ground Up
46mFull-Text Search Internals with Elasticsearch
01h 28mBuilding a Semantic Search Engine