AI-powered Semantic Search; A story of broken promises?
summary
Semantic vector search often fails out of domain: models trained on MS MARCO: 9M passages, 500k labels, 3,600 test queries give great in-domain results but dense embeddings frequently lose to BM25 across many benchmarks. BERT-style fine-tuning yields huge gains—BERT lifts SOTA ~30% in a week—but cross-encoders are costly and by-encoders need domain adaptation (document-to-query generation). Tooling like Vespa enables hybrid retrieval and claims 1B vectors on one node <10ms, while raw pre-trained embeddings (e.g., OpenAI) often match BM25 only.
AI-written summary · about
chapters (17)
- 0:00Introduction to search
- 0:28Keyword vs semantic search
- 1:24Tooling and technologies
- 3:11Talk overview and themes
- 4:16Supervised learning basics
- 5:57The MS MARCO dataset
- 8:41Deep learning breakthroughs
- 10:24BERT for search ranking
- 11:46Pre-training and fine-tuning
- 13:22Vector vs cross-encoder
- 15:24In-domain vs out-of-domain
- 16:52Zero shot capabilities
- 18:24Generalization challenges
- 21:01Key takeaways for search
- 23:04Benefits of vector search
- 24:36Vespa platform solution
- 25:55Q&A session
