Importance of Transactional Metadata (Or How We’re Improving Upon Gossip)
summary
Replaces gossip for cluster metadata with transactional cluster metadata implemented as an ordered change log of transformations, identified by monotonically increasing epochs and stored on a replicated cluster metadata service (CMS). Uses a progress barrier to force quorum acknowledgements between steps so concurrent bootstraps no longer produce non‑intersecting majorities and transient writes are prevented. Implementation supports resumable bootstrap/decommission, structural‑sharing state snapshots, hundreds of hours of simulator testing (Harry + custom simulators), a two‑phase upgrade path, and backward compatibility with tokens.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 1:00Overview
- 2:03Gossip
- 7:05Token Ownership
- 8:59Single Point of Reference
- 10:57Transient Loss of the Right
- 12:47Other Problems
- 13:32Requirements
- 14:34Implementation Details
- 15:47Transformations
- 19:17Metadata State
- 20:31Metadata Elements
- 21:50Metadata Consistency
- 22:11Metadata Bootstrapped
- 22:35Prepare Join
- 22:58Add X
- 23:34Progress Barrier
- 25:20Midjoin
- 25:40Manifest
- 27:00Recovery
- 29:27Elasticity
- 29:48Transactional Metadata
- 30:50Recap
- 32:20QA
- 38:49Upgrade Process
related talks
47mDatabase Resiliency
41mTransactions: myths, surprises and opportunities
47mBeing transactional in 2019 - what does this actually mean?
43mBeing Transactional in 2019 — What Does That Actually Mean?
19mBuilding a Performant Configuration-Store Layer
31mHow to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
46mJourney of metadata
59mDistributed Abstractions