Fine Grained Replicated State Machines for a Cluster Storage System
summary
Design treats every key-value pair as a separate replicated state machine and co-designs a Paxos-style consensus with an LSM-based store to drop per-key log history and keep only proposal/accept metadata. Evaluation characterizes 980 production clusters and runs controlled tests on four servers (12 cores, two SSDs each), showing 2.3x throughput versus shard-per-log systems by reducing head-of-line blocking, improving batching across SSDs, and enabling leader-only reads via background scans and minimal per-key consensus state.
AI-written summary · about
chapters (4)
- 0:00Introduction
- 0:16System architecture overview
- 0:30Design challenges and solution
- 10:35Implementation and evaluation
related talks
39mTrust Deterministic Execution to Scale & Simplify Your Systems
21mDDD and FSM: tackling complexity with state machines
13mSharding the State Machine: Automated Modular Reasoning for Complex Concurrent System
16mSwiftPaxos: Fast Geo-Replicated State Machines
52mThe Computer Science behind a modern distributed data store
44mDistributed systems and scalability rules
24mStronger Semantics for Low-Latency Geo-Replicated Storage
45mDo Distributed Systems they said. It'll be fun they said.