To Shard or Not to Shard
summary
Argues most apps avoid sharding because a single MySQL box routinely reaches 100,000 qps and a 3M DAU example maps to ~31k peak queries, so sharding is often premature. Lists concrete pains: development complexity, operational split/reshard work, schema rollout headaches and weird failure modes. Recommends delaying via replication, caching (Memcached/Redis), queues (Redis/RabbitMQ/Kafka), materialized tables, or clustering like Percona XtraDB Cluster and proxies/Vitess/MaxScale; notes sharding’s real benefits: extreme scale, isolation and cost tuning.
AI-written summary · about
chapters (35)
- 0:00Story
- 1:35First things to decide
- 2:17Single MySQL Can Do
- 3:41Let's do some math
- 4:48Avoided Sharding
- 5:32Sharding Pains
- 7:37MySQL Sharding
- 8:27Can't Avoid?
- 9:18Strategies to Delay Sharding
- 9:29Architecture
- 10:33Functional Partitioning
- 12:19Replication
- 13:25Caching
- 16:57Queueing
- 19:06Beyond MySQL
- 19:40Optimize!
- 20:18Hardware
- 23:36Environment
- 24:56Configuration
- 25:57Sharding - When?
- 26:41Architectural Runway
- 28:27Capacity Planning
- 29:53Benefits of Sharding
- 31:30Avoid other complexities
- 32:30Isolation
- 33:43Costs
- 34:45When to Shard Summary
- 36:23Sharding Questions
- 36:34Sharding Level
- 37:46Sharding Key(s)
- 39:29Sharding Unit
- 41:08Sharding HA
- 42:01Sharding Technology
- 45:15In Summary
- 45:41Thank you!
related talks
28mSharding: DIY or Out of the Box Solution?
32mSharding for mission-critical, internet-scale apps
27mDistributed MySQL Architectures: Past, Present and Future
24mWhen and How Should you Shard?
46mDatabase Sharding the Right Way Easy, Reliable, and Open source
37mSharding: Growing Systems from Node-scale to Planet-scale
33mFive Sharding Data Models and Which is Right
23mSharding for hyperscale and geographically distributed relational databases