Building an Image Search Engine with Generative Computer Vision Models
summary
Walks through building a working image search pipeline that pairs CLIP embeddings with an approximate vector database and nearest-neighbor search, includes a hands-on coding demo and a live run that executes locally. Shows how images and captions become vectors, how cosine distances cluster items (cat vs kitten vs dog examples with distances around 0.6 and 0.1), and how optional steps like clustering or using diffusion features change recall. Covers indexing, retrieval latency tradeoffs, and when to train custom embeddings.
AI-written summary · about
chapters (12)
- 0:00Image search engines intro
- 6:29Search systems core concepts
- 14:14Query processing
- 22:02Relevance scoring & data
- 33:06CLIP model architecture
- 45:10CLIP embeddings similarity
- 51:26CLIP embeddings demo
- 1:00:48Search pipeline with CLIP
- 1:11:17Models to database
- 1:15:25Pre-trained vs custom CLIP
- 1:21:35Implementation steps & demo
- 1:24:40Q&A
related talks
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
01h 30mBuilding a Search Engine
26mImage search without image captions
01h 08mGoing multi-modal
28mVisual Search
48mEnter the Brave New World of GenAI with Vector Search
59mContextual search with vector search: exploring your options with open source tools
43mThe Search for a New Visual Search, Beyond Language