Under the Hood: Key Algorithms Powering Vector Search
summary
Covers HNSW, LSH and product quantization with concrete tradeoffs: HNSW + PQ yields fastest searches with ~0.86 recall after ~50s build time while flat scan is 100% recall but ~4s query and no index, PQ alone drops recall to ~0.43, LSH is fast to build but low recall. Demonstrates FAISS benchmarks and claims a practical 2,000x memory reduction (1M×1024-d float32 → ~16–17 MB with PQ). Explains index construction, parameters, and when combined approaches succeed.
AI-written summary · about
chapters (11)
- 0:00Vector search algorithms intro
- 1:21Why vector search matters
- 3:54Nearest neighbor challenge
- 4:58Four vector indexes types
- 5:39Locality sensitive hashing deep dive
- 9:33Product quantization for memory
- 18:21Hierarchical navigable small worlds
- 24:47Building HNSW step by step
- 26:12Benchmarking index performance
- 28:42HNSW with product quantization
- 29:25Q&A
related talks
44mVector Search Beyond the Hype: Real-World Implementation and Performance Insights
43mOpenSearch and the Path to a Trillion-scale Vector Database
32mRetrieval Augmentation and Semantic Search at Scale
40mUnder the hood of vector search with JVector
54mDissimilarity search...
34mThe Challenges of Making Vector Search Billion-scale
01h 09mVector Search: Ask Me Anything!
33mBeyond Benchmarks: Real-World AI Search Performance in OpenSearch (Lucene Vs Faiss Vs JVector)