Providing Flexible Database Consistency Levels with Manhattan at Twitter
summary
Manhattan adds a replicated sequential log to its quorumbased KV path so writes can be ordered per shard, yielding per-shard linearizability and the ability to mix eventual and strong operations in one cluster. Implementation uses the Twitter distributed log (built on a patched BookKeeper), costs a few extra milliseconds locally and an order of magnitude more across regions, keeps failure isolation to a shard, requires catchup on topology changes, and needs strong observability and tooling to handle replay or apply failures.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:22USING MANHATTAN
- 2:22ADOPTION
- 3:21BUILDING NEW FEATURES
- 5:48ARCHITECTURE: DATA MODEL
- 6:39ARCHITECTURE: PARTITIONING
- 6:56ARCHITECTURE: MESSAGING
- 7:38ARCHITECTURE: CONSISTENCY
- 9:17BENEFITS OF EVENTUAL CONSISTENCY
- 10:48WHEN IT'S NOT ENOUGH
- 13:17ADAPTING ARCHITECTURE
- 21:10RESULTS
- 26:23A DIFFERENT CONSISTENCY MODEL
- 31:43A DIFFERENT TOPOLOGY TRANSITION
- 34:10A DIFFERENT DEFINITION OF TIME
- 37:29WHAT THE USER SEES
related talks
30mThe Distributed Database Behind Twitter
35mStorage, Then and Now
29mModern Database Architecture: From SQL to NoSQL & Distributed SQL
29mMulti Active-Active and disaster recovery with MongoDB database center
30mThe Distributed SQL Database Behind Twitter
12mFault-Tolerant Replication with Pull-Based Consensus in MongoDB
24mStronger Semantics for Low-Latency Geo-Replicated Storage
40mDistributed Commit Log: Application Techniques for Transaction Processing