Unlocking the Power of Hybrid Search: A Deep Dive into Python-Powered Precision
summary
A concrete build story: Python glue wires Scrapy + LlamaIndex for chunking, Gemini (after trying Nomic AI) for 768‑dim embeddings, and Aerospike as a NoSQL store with an HNSW vector index. The index covers about 6,000 chunks from ~2,000 pages, cost about $2 to embed ~18,000 vectors, and queries average 80 ms average latency. Keyword side uses SpaCy + inverted index + BM25; results merge with reciprocal rank fusion. Shows code-level choices, tradeoffs, and a working site.
AI-written summary · about
chapters (12)
- 0:00Hybrid search project intro
- 2:54Why build a better search engine
- 6:02Scraping and preparing data
- 9:58Vector embeddings for semantic search
- 16:29Storing vectors in Aerospike
- 17:56Querying vectors for semantic search
- 19:22Building keyword index
- 21:56Ranking with BM25 and fusion
- 25:12Python backend with FastAPI
- 26:38Frontend integration demo
- 28:22Future Aerospike search plans
- 29:57Q&A
related talks
28mA Deep Dive into Python-Powered Precision and Scalability
45mRevolutionizing Search with Power of Neural Sparse and Dense Vectors
45mBehind the hype: managing billion-scale embeddings in Elasticsearch and OpenSearch
01h 28mBuilding a Semantic Search Engine
48mUnlocking Next-gen Semantic Search With OpenSearch 3.x
27mUnified AI & Relational Power: Scalable Distributed Vector Search with
22mBuilding a Multimodal Search Engine
19mSemantic Search for the Rest of Us with DuckDB