pgvector Newbie to Expert
summary
Practical roadmap for running pgvector on PostgreSQL: install the extension, store embeddings (demo loads 86 session chunks using Amazon Titan and LangChain), choose data types (vector, vector_half, sparse, bit), and pick indexes (HNSW vs IVF-flat) with distance metrics (cosine, Euclidean, inner product). Key numbers and tradeoffs: HNSW supports dynamic inserts and better recall but larger build time and memory, IVF-flat is faster to build, indexable dimension limits require quantization for very high dims, EF_search and max_scan_tuples tune recall vs QPS, and role‑level security, toast storage, partitioning, and partial indexes affect correctness and performance.
AI-written summary · about
chapters (21)
- 0:00Session introduction
- 1:25Generative AI applications
- 4:20Retrieval augmented generation
- 7:51Vector embeddings role
- 9:21PostgreSQL as vector DB
- 10:45PG vector install & usage
- 13:14Vector data types trade-offs
- 15:04Exact vs approximate searches
- 16:54Similarity search algorithms
- 18:03Indexing IVF flat & HNSW
- 21:41Production best practices
- 22:23Security & role filtering
- 24:14Storage TOAST config
- 28:27Scan performance comparison
- 30:12Quantization for vectors
- 33:42Filtering challenges
- 37:37Indexing & partitioning
- 39:23Index tuning
- 40:25Live demo setup
- 46:42RAG pipeline evaluation
- 48:46Q&A
related talks
52mVector search is now boring, but PostgreSQL has ways to go
53mVectors: how to better support a nasty data type in PostgreSQL
27mpgvector for Python developers
48mVectors are the new JSON
25mVector data in Postgres - how's it different from "normal" data?
41mPostgreSQL for AI. Real Vector Search in PostgreSQL
04mSupercharging GenAI PostgreSQL Vector Search Optimization
44mScaling Semantic Search in PostgreSQL