Milvus: Scaling Vector Data Solutions with Gen AI
summary
Milvus is a distributed, cloud-native vector database built on MinIO and Kafka and designed to scale to billion-plus collections (the project started after a customer needed to handle five trillion vectors). Architecture separates concerns into query node, data node, index node, uses a pub/sub write log for tunable strong or eventual consistency, orchestrates in-memory/HDD indexes, and supports IVF and HNSW with scalar/product quantization. Primary wins reported are horizontal scale, near-100% HNSW recall, and practical RAG and multimodal search at large scale.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:08Vector databases needed
- 7:32Vector similarity and embeddings
- 12:54Indexing methods
- 19:13Data structure and metadata
- 20:59Gen AI use case
- 22:43Example projects and code
- 25:33Vector vs SQL/NoSQL
- 26:38Milvus architecture
- 31:36Summary and Q&A
related talks
37mMilvus: Scaling Vector Data Solutions with Gen AI- 37mMilvus: Scaling Vector Data Solutions with Gen AI
30mMilvus Vector Database: Unlocking the Future of Open Source Vector Databases
29mThe Evolution of Milvus: A Cloud-Native Vector Database
35mMilvus 2.0: Building the World's Most Advanced Open-source Vector Database
31mMilvus 2.0: A Vector Database With Cloud-Native Architecture
51mUnleashing the Potential of Cloud-Native Vector Databases
40mEvolution of Milvus Cloud-Scalable Vector Database