Sharding and Things We'd Like To See Improved
summary
Says MongoDB 5.0 can reshard a collection with a new shard key but practical problems remain: the config metadata records chunk ranges but contains no chunk data-size, TTL deletes produce many empty chunks and the balancer only evens chunk counts not bytes. Chunk migrations create short collection-level metadata locks and rebalancing can take days while hot shards stay hot. Mentions refine/reshard commands, recommends balancer awareness of chunk size, automatic empty-chunk merging and better cluster-wide observability.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:50Vertical Scaling
- 1:13Horizontal Scaling - Reads
- 1:57Horizontal Scaling - Writes
- 2:43Scaling per tenant
- 5:15Scaling with MongoDB - Shards
- 7:57Scaling with MongoDB - Config Servers
- 10:52Scaling with MongoDB - mongos
- 12:00Choosing shard key
- 16:03Ranged based sharding
- 20:42Balancer
- 26:54Refining a shard key
- 27:23Resharding a collection
- 28:10Balancing
- 30:32Cluster metadata
- 32:32Merging empty chunks
- 35:29Migrating chunks
- 37:07Observability
- 41:08Summary
related talks
24mWhen and How Should you Shard?
47mSharding in MongoDB
32mA New Sharding Architecture for MongoDB
51m5 boring performance improvements in MongoDB 3.4 that matter
30mUnderstanding Bottlenecks & Conquering them
32mThe New MongoDB Sharding Architecture
34mScaling Applications with MongoDB: Best Practices and Challenges
10mEverything You Know About MongoDB is Wrong