Zero-Downtime Rebalancing and Data Migration of a Mature Multi-Shard Platform
Talkat SREcon19 Europe/Middle East/Africa · on Wednesday, 2019-10-02· at 16:45· Track 1: The Liffey B·
summary
Shopify describes a production toolkit that uses the MySQL binlog event stream to copy and replay row changes, rebalancing hundreds of MySQL shards and migrating over 800,000 tenants to Google Cloud with no tenant-facing downtime. It implements per-tenant shared-exclusive locking to bound the write-free cutover (median ~2.5s), batch copy + incremental replay, and a fingerprint verification (hashes + GROUP_CONCAT) that speeds integrity checks ~5×. The system is open-sourced and actively runs in production.
AI-written summary · about
chapters (9)
- 0:00Introduction to sharding
- 3:09Data structure and routing
- 5:17Primary key generation
- 9:10Layer 7 request routing
- 11:09Shard rebalancing challenges
- 17:25Shared exclusive locking
- 22:46Minimizing downtime
- 27:48Data integrity verification
- 33:24Orchestration and closing
related talks
22mReliable Data Replication
39mScaling Shopify's Multi-Tenant Architecture across Multiple Datacenters
19mMove to the Cloud with Zero Downtime
41mDemocratizing MySQL: From Cloud Managed to Kubernetes Managed
34mZero Downtime Database Migrations and Deployments
24mMigrations under Production Load: How to Switch...
46mEdgestore Multi-tenancy & Isolation
09mRuby startup demo – Sam Lambert