Sharding: DIY or Out of the Box Solution?
summary
Describes concrete outcomes: a DIY sharding reuses existing ProxySQL rules, adds per-shard connections and ProxySQL per-shard connections, shards by uid with 10‑day retention, copes with write pressure but causes duplicated rows during primary outages and heavy uid recalculation load; Vitess (deployed on VMs in the proof of concept) requires tuning, automation and learning but offers key features like materialized views, easier shard splits and faster scale‑out. Simple one‑cluster problems favor DIY, multiple clusters favor Vitess.
AI-written summary · about
chapters (23)
- 0:00Intro
- 3:11What is (database) sharding?
- 3:59What is functional sharding?
- 5:30What is horizontal sharding?
- 8:57Problematic replication on short-term storage database
- 10:24Possible solutions
- 12:21Sharding algorithm
- 14:15How does that look from the application side?
- 14:43What if a shard becomes unavailable?
- 16:17Did it work out as planned?
- 18:50The Archive challenge
- 19:33The Archive solution
- 19:46What boundaries were sett
- 20:19Why did we consider Vitess?
- 20:59Choosing the right solution for the Archive
- 21:35What did we encounter during the PCT
- 22:27How do we perceive them?
- 23:57Operational: schema changes
- 24:36Operational scaling out
- 25:13Operational: MySQL upgrades
- 25:43Operational: sharding platform upgrades
- 26:02Running Vitess on "iron"
- 26:44Is Vitess saving you time?
