Advancing Search and Observability at Uber with OpenSearch
summary
Uber extends OpenSearch with production-scale semantic vector search using embeddings, a two-phase pipeline (cosine retrieval to winnow billions to thousands, then learn-to-rank reranking) and real-time query embedding, supporting 50 billion queries/day across 150+ clusters. For metrics Uber builds a time-series engine inside OpenSearch with new data structures, compression and Prometheus-compatible query support plus pluggable M3 integration, targeting 5B+ metrics/sec and 160B cardinality. Work is being upstreamed but full migration from a Lucene/M3 homegrown stack is still in progress.
AI-written summary · about
chapters (7)
- 0:00Search at Uber
- 1:43Transition to semantic search
- 3:00Semantic search architecture
- 5:48Scale and observability
- 7:29Next-generation metrics
- 9:34Time series engine
- 10:56Collaboration and future
related talks
18mAdvancing Search at Uber with OpenSearch
24mVector Search at Uber
48mUnlocking Next-gen Semantic Search With OpenSearch 3.x
26mAdvancing Multi-Modal Search Capabilities in Search Pipeline
26mOpenSearch and the Path to a Trillion-s...
41mText to Context: How We Introduced Hybrid Search
32mRetrieval Augmentation and Semantic Search at Scale
24mRediscover Your Data with a New Multimodal Search Capability