Extracting insights from unstructured data
summary
Elastic Stack (Elasticsearch + Kibana) stores text, GPS, images and vectors and enables vector search with k‑nearest‑neighbor to find similar embeddings; BM25 lexical search still works and can be combined via hybrid retrieval with boost weights (example: boost 0.9 vs 0.1). The talk cites industry numbers—80–90% of data is unstructured and <20% is analyzed—and shows practical steps: import PyTorch models from Hugging Face into Elasticsearch (via the elan Python client), deploy to an ML node (import ~20 minutes) and run semantic + image searches.
AI-written summary · about
chapters (15)
- 0:00Introduction and agenda
- 1:08What is Elastic and Elasticsearch
- 2:36The role of data in modern life
- 6:51Overview of the Elastic Stack
- 8:56Challenges of unstructured data
- 10:21What is Vector search
- 14:37Lexical vs semantic search
- 17:50Hybrid retrieval explained
- 20:41Generating text embeddings
- 22:07Deploying NLP models in Elasticsearch
- 24:18NLP tasks beyond text embeddings
- 25:00Image similarity with Vector search
- 27:47Combining text and image data
- 28:56Searching any unstructured data
- 29:37Summary and next steps
related talks
47mLatest innovations from Elasticsearch: Drive speed, scale & relevance (PRT266)
15mMastering RAG and Vector Search with Elastic
31mTaking AI/ML to the next level: Text!
19mVectorize Your Open Source Search Engine
42mResearch in the Age of AI
38mSearch: A New Era
30mSearch on Steroids — A Story of Semantics
37mLexical, Vector & Hybrid Search with Elasticsearch