The Secret Life of Distributed Systems
summary
Examines designs and tradeoffs for exabyte-scale distributed systems, with concrete coverage of replicated state machines using Raft to turn logs into durable state, synchronous vs asynchronous replication choices, and quorum tuning. Describes storage patterns used to build BigTable-style keyspaces and S3-backed SQL (Aurora/Neon), cites targets like 350 trillion files and 100M requests/sec, and compares streaming histories (Kafka versus Redpanda/Northguard) while noting observed bottlenecks and which solutions worked in practice.
AI-written summary · about
chapters (11)
- 0:00Distributed systems intro
- 2:07True distributed systems
- 4:58Log abstraction
- 10:37Data replication
- 18:38Quorums for reliability
- 23:53Broadcast and consensus
- 31:15Reliable query store design
- 41:43Distributed file systems
- 47:05Database evolution
- 53:04Modern distributed logs
- 55:29Q&A
related talks
30mDistributed system design
11mExplaining distributed systems like I'm five
29mFollowing the Data: A Deeper Look into Distributed Applications
57mScaling the CAP Theorem for Modern Distributed Systems
23mDesigning Robust and Scalable Distributed Applications
46mBuilding high available systems with Akka Distributed Data
39mArchitectural design patterns applied to Distributed Systems
38mHow To Build a Distributed System (And Should You?)