OpenSearch Migrations: Modernize Without Worry
summary
Migration Assistant automates multi‑version upgrades by combining reindex from snapshot to backfill historical data and capture and replay (Kafka-backed) to keep live traffic in sync, with JavaScript transformations for breaking changes like multi‑type mappings. Customers move 4 TB in 7 hours, 5.5 TB in a day, and large cases hit ~100 TB/day using 150 workers for a 450 TB migration; sample AWS cost is ~$5,000 to move 100 TB over three weeks. Kubernetes + Argo workflows and an API are coming.
AI-written summary · about
chapters (16)
- 0:00OpenSearch migrations intro
- 1:04Why migrations now
- 2:06Reindex from snapshot
- 3:34Live data capture and replay
- 5:40Infrastructure and cost
- 7:26Validation and confidence
- 8:55Customer success stories
- 10:22Case study Octis credit intelligence
- 14:53Brand sharding upgrade
- 17:43Amazon Ads data migration
- 20:57Telecom policy challenges
- 21:39Elastic Cloud to OpenSearch
- 22:22Telecom daily data filtering
- 23:28Migration Assistant roadmap
- 27:05API and UI for migrations
- 28:57Resources and Q&A
related talks
38mMigrating from Self-Managed ElasticSearch to OpenSearch
25mRunning OpenSearch in the Real World: Migration, Mistakes and Maturity
36mScaling Log Management: PB-Scale Migration From Elasticsearch To...
31mFrom Atlas to EKS How to Stay Sane and Save 45 During a Large‑Scale MongoDB Migration
10mFrom custom Solr to out-of-the-box OpenSearch
31mMigrating from Self managed Elasticsearch to OpenSearch & How Indias Fantasy Sports Giant Leveled Up
37mDatabase Modernization at Cloud Scale: Minimizing Risk and Disruption
44mScaling OpenSearch at Uber: Operating Resilient, Fault-Tolerant, High-Performance Clus...