InnoDB to MyRocks Migration in Main MySQL Database
summary
Facebook migrates massive, latency‑sensitive MySQL shards from InnoDB to MyRocks (RocksDB inside MySQL), and reports reduced DB size by half and ability to consolidate two instances per host. Migration uses a primary‑key sorted fast bulk loader that bypasses RocksDB write‑amplification, extensive data‑consistency checks (index counts, statistics, consistent snapshot comparisons) and a shadow traffic testing framework, plus careful compaction and crash‑recovery tuning; the engine runs in production at Facebook.
AI-written summary · about
chapters (29)
- 0:00Intro
- 2:23"Main MySQL Database" at Facebook
- 6:25Write Amplification
- 8:37B+Tree Fragmentation
- 10:14Compression
- 13:39How LSM/RocksDB works
- 15:43What is MyRocks
- 20:05MyRocks goals (more details)
- 21:34MyRocks features
- 23:56Relationship between MySQL PE and SWE
- 27:16MyRocks migration - Technical Challenges (2)
- 28:46Creating first MyRocks instance without downtime
- 29:39Faster Data Loading
- 31:16Data migration steps
- 32:26Data Verification
- 32:44Verification tests
- 35:39Shadow traffic tests
- 36:40Creating second MyRocks instance without downtime
- 38:08Promoting MyRocks as a master
- 40:32Master crash safety settings
- 42:37Notable issues fixed during migration
- 43:38Preventing stalls
- 44:41When write stalls happen
- 45:46What happens on write stalls
- 46:36Mitigating Write Stalls Speed up compactions
- 48:04Monitoring
- 48:17SHOW ENGINE ROCKSDB STATUS
- 48:39SHOW GLOBAL STATUS
- 48:49Summary
related talks
12mMyRocks at Facebook and a Roadmap
47mLessons Learned from Our Main Database Migrations at Facebook
39mLessons Learned from Our Main Database Migrations
51mMyRocks deep dive and production deployment at Facebook
52mRocksDB Performance and Reliability Practics
27mMyRocks: best practice at Alibaba - MySQL Storage Engine
45mMyRocks Table Engine for MySQL
57mRocksDB