Harnessing the power of LLM text embeddings
summary
Clary builds Co‑pilot to extract insights from sales call transcripts using text embeddings; experiments compare vanilla OpenAI embeddings, a learned projection matrix over those vectors, and full fine‑tuning of open models. Using a few thousand query–document pairs and hard negatives the projection raises search accuracy from about 67% to 92.5%, while cluster‑based fine‑tuning creates a clearly separated similarity distribution. Models run on hosted infrastructure (previously SageMaker) now on Kubernetes with GPUs for latency and cost control.
AI-written summary · about
chapters (14)
- 0:00Introduction to talk
- 1:55What is Clary and co-pilot
- 6:06How co-pilot uses LLM embeddings
- 10:02From lexical to semantic search
- 13:13Word and sentence embeddings explained
- 18:25Embeddings in co-pilot product
- 21:56Why customize embeddings
- 23:42Customization methods overview
- 26:07Search vs similarity tasks
- 28:13Building evaluation datasets
- 30:59Metrics for embedding performance
- 32:02Experiment results and insights
- 36:45Future work and challenges
- 37:48Closing and Q&A
related talks
09mFine-tuning embeddings 4 vector search
39mExperiments with OpenSearch and AI
01h 08mGoing multi-modal
15mSearch Engineering Reimagined: Leveraging LLM in ML Workflows
31mExploring LLMs across clouds – A Year in the Life
31mText and Code Embeddings
51mQuery Understanding With Large Language Models: Techniques and...
16mEmpower AI: Building Smarter Solutions with Embeddings