How We Survive Split-Brain and Keep Writing Data via the S3 Protocol
summary
An object storage system that uses the S3 protocol is deployed across multiple geographically distributed sites and must continue accepting writes even during complete network partitions and ambiguous split-brain situations. The talk explains how the storage enforces complex data placement and retention rules across sites and the techniques used to tolerate partitions and reconcile or continue operations despite inconsistent network connectivity.
AI-written summary · about
chapters (7)
- 0:00Introduction to Frost FS
- 10:02S3 protocol issues
- 11:28Naive solutions failures
- 13:57CRDT trees for consistency
- 22:26System demo
- 32:08Split-brain solution and results
- 44:04Q&A
related talks
50mA Homegrown S3 Server: Challenges of Building Stable Storage
27mAmazon S3 Strong Consistency
22mJoining Dozens of Data Streams in Distributed Stream Processing Systems
28mArchitecting for high availability on Amazon S3
35mSystems @Scale Summer Q&A Week 3
34mDistributed systems 102
31mThe Design of a Modern Distributed Object Storage
23mRobustness in the Salus Scalable Block Store