Similarity Search for Product Matching @ Semantics3
summary
Semantics3 reframes deduplication as similarity search, trains a Siamese network with a boolean match label to embed products, and swaps Elasticsearch for Facebook’s FAISS index. The production system indexes ~60 million vectors (full index <4GB), yields ~390 ms median read at k=100 (19 ms amortized with batching), and cuts dedupe cost from ~$300 to $30 per 100k dedup while keeping a small accuracy drop from vector quantization. It produces candidate neighbors for a downstream exact-match model.
AI-written summary · about
chapters (19)
- 0:00Start
- 2:24Overview
- 3:01Product Matching
- 4:52What is a match
- 5:45What is not a match?
- 6:29How do we go about solving this?
- 7:57Needle in a haystack
- 8:56Reality
- 10:41Can't we just use the structured data?
- 13:37Peeking in
- 14:38Last layer of categorizer
- 16:20Siamese Twinning Tuning
- 17:22Similarity Search
- 19:44We gave it a spin
- 20:59Did we really need a database?
- 21:52But, what about writes?
- 22:37Key benchmarks
- 23:29Lessons
- 24:51Questions?
related talks
32mBuilding semantic search using Deep Learning
30mSearch on Steroids — A Story of Semantics
33mA Practical Approach To Semantic Search
22mHarnessing image and text data for product matching
22mThe Capabilities of Semantic Search
14mSearch on Steroids
01h 01mVector Similarity Search in Spring
31mAdvanced search for your e-commerce website