GenAI, vector/semantic/hybrid search, RRF, NLP, (...) and other buzzwords
summary
Connects OpenAI GPT-4o to Elasticsearch, generates vector embeddings (384 dims) from indexed JEP documents, and uses kNN + hybrid boolean scoring to build a simple RAG pipeline; demo fetches JEP 461 and asks the model what a stream gatherer is, returning the correct structure (integrator mandatory, initializer/combiner/finisher optional). Explains Lucene-backed vector storage, tuning boost vs hybrid scoring, and recommends RRF (reciprocal rank fusion) to combine classic and vector ranks to reduce hallucination and ease migration.
AI-written summary · about
chapters (6)
- 0:00Introduction and AI buzzwords
- 13:29From training to real use cases
- 21:19Vector search and embeddings
- 30:08Hybrid search and scoring
- 37:58Demo of RAG implementation
- 39:46When to use AI and closing
related talks
34mEnter the Brave New World of GenAI with Vector Search
48mEnter the Brave New World of GenAI with Vector Search
57mEnter the Brave New World of GenAI with Vector Search
28mUse OpenSearch as Your Vector Search Database To Optimize GenAI Accuracy!
44mVector Search Isn’t Enough
45mMaking semantic search & RAG real: How to build a prod-ready app
15mMastering RAG and Vector Search with Elastic
33mEnter the Brave New World of GenAI with Vector Search