Full-Text Search Demystified

on YouTube ↗55m

" summary

Explains how Elasticsearch (backed by Lucene) turns JSON documents into searchable data: HTML stripping, tokenization, lowercasing, stopword removal and stemming, then an inverted index with term frequency and positions that enables phrase, fuzzy and n‑gram searches and bucketed aggregations. Describes scoring (TF, IDF, field length norm, vector space) and limitations: mappings are fixed (reindex to change), writes are near real-time (~1s) but can be tuned, use the bulk API for fast ingestion, and Java clients include XContentFactory and community Spring Data Elasticsearch for ODM-style access.

AI-written summary · about

" chapters (11)

1. 0:00Introduction and context
2. 2:13Search versus databases
3. 5:55Elasticsearch basics
4. 8:57The indexing process
5. 14:42Inverted index and querying
6. 20:12Scoring and relevance
7. 27:30Advanced search techniques
8. 34:01Java and ODM integration
9. 37:26Handling data models
10. 43:12Conclusion and summary
11. 47:17Q&A session

" related talks

• 46mFull-Text Search Internals with ElasticsearchPhilipp Krenn · code.talks commerce 2018
• 35mElasticsearch - Beyond Full-text SearchAlex Reelsen · GOTO Aarhus 2013
• 26mFrom __icontains to searchHonza Kral · DjangoCon US 2014
• 37mAlgorithms & data-structures that power Lucene & ElasticSearchAdrien Grand · Berlin Buzzwords 2015
• 16mWhy we built ElasticSearchShay Banon · dotScale 2013
• 25mApache Lucene: From Text Indexing to Artificial IntelligenceLucian Precup · Devoxx France 2024
• 47mMake sense of your (big) data using ElasticsearchHonza Kral · PyData Berlin 2014
• 45mAnything New in Search? What If I Find It?Mikhail Khludnev · Joker 2022