From Atlas to EKS How to Stay Sane and Save 45 During a Large‑Scale MongoDB Migration
summary
Migrating seven clusters (small replicas up to three sharded clusters with ~5 TB per shard) uses Percona Server for MongoDB on Kubernetes via Percona operator and Argo CD, with initial bulk copy by mongosync and change capture shifted to Kafka CDC after mongosync’s CDC fails; enabling pre/post images forces full documents into the oplog so Kafka can keep up. Migration runs under a two‑hour maintenance window, moves six of seven clusters before Black Friday, finishes Dec 15, and cuts costs 45% (~$0.5M/yr) while adding 24/7 operational responsibility.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:09Why Atlas stopped working for our use case
- 4:45Choosing the right platform for self-managed MongoDB
- 7:15Percona Server selection
- 9:06Migration strategy
- 12:22Critical problems we faced at production scale
- 15:38Infrastructure solution
- 17:26Results and life after the cut-over
- 19:15Self-managed trade-offs
- 20:19Q&A
- 23:34Q&A: Time to migrate five terabytes
- 24:18Cloud/network choices
- 25:02Operational overhead
- 26:09Monitoring setup
- 27:20Dual-write vs oplog
- 29:54Cut-over downtime
related talks
01h 00mCross-Cloud Migrations With Databases on Kubernetes
37mHow We Sharded and Migrated a 20 TB MongoDB Without Downtime
42mOpenSearch Migrations: Modernize Without Worry
27mDIY DBaaS with Percona Kubernetes Operators
40mData Migration Lessons from the Trenches
25mCourting Mongo, committing to Postgres: Our data love story!
34mLarge Scale NoSQL DB Migration Under Fire
24mMoving MongoDB to the Cloud: Strategies and points to consider