How We Sharded and Migrated a 20 TB MongoDB Without Downtime
summary
Avito needed to move multiple sharded MongoDB clusters (each 1–5 TB, totaling ~20 TB) from large fast bare-metal servers to smaller slower Kubernetes nodes (~300 GB). The talk explains the migration and sharding strategy used to perform this without downtime, and discusses practical pitfalls encountered during the process.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 5:01Growth problem and Kubernetes move
- 9:17Shard count calculation
- 13:54Migrator choice for resharding
- 21:20Data imbalance issue
- 24:29Changing shard key
- 28:31Results and conclusion
- 29:14Q&A
related talks
31mFrom Atlas to EKS How to Stay Sane and Save 45 During a Large‑Scale MongoDB Migration
40mData Migration Lessons from the Trenches
24mMoving MongoDB to the Cloud: Strategies and points to consider
41mHow to Survive Under Load with 100TB in an Unsharded MongoDB
35mClusters as Cattle: How to Seamlessly Migrate Apps Across Kubernetes Clusters
32mA New Sharding Architecture for MongoDB
42mOpenSearch Migrations: Modernize Without Worry
43mCRIMS: The case of the DB change that left no trace