Vectors are the new JSON
summary
PostgreSQL already stores vectors via the pgvector extension, adding a vector type plus IVF‑flat and HNSW indexes; common embeddings are 1,536‑dim (~6 KB each) so a million rows is ~6 GB and hits TOAST and page limits. HNSW gives better recall/perf tradeoffs controlled by M and EF_construction/EF_search, IVF‑flat builds faster but needs tuned probes and memory; filtering often forces scans so use partial indexes, partitioning, parallel builds or quantization to shrink storage.
AI-written summary · about
chapters (19)
- 0:00Introduction to vectors
- 1:07JSON’s evolution in databases
- 3:15What are vectors?
- 4:39Vectors and AI models
- 7:48Vector role in RAG workflow
- 10:16Vector challenges in databases
- 13:46Approximate nearest neighbor search
- 15:54Vector storage considerations
- 17:43Postgres and vector support
- 19:08Introducing pgvector
- 20:14Performance benchmarks
- 22:19Distance operations in pgvector
- 24:26Indexing methods IVF and HNSW
- 28:20Storage and TOAST impact
- 32:36HNSW index tuning
- 37:52IVF flat indexing
- 40:39Filtering in vector queries
- 43:47Future of pgvector
- 46:56Conclusion and Q&A
related talks
53mVectors: how to better support a nasty data type in PostgreSQL
25mVector data in Postgres - how's it different from "normal" data?
51mVector data in Postgres: Size, TOAST, Filters and Performance
52mVector search is now boring, but PostgreSQL has ways to go
52mpgvector Newbie to Expert
44mScaling Semantic Search in PostgreSQL
48mFrom Text to Meaning: pgvector Transforms PostgreSQL Search
41mPostgreSQL for AI. Real Vector Search in PostgreSQL