Trust Deterministic Execution to Scale & Simplify Your Systems
summary
Claim: making core business logic a deterministic single-threaded finite state machine and replicating it via a replicated log lets the system rerun requests at the edges instead of shipping bulky events. Built a blazing fast open source raft cluster, a mirrored shadow environment that streams production logs, and local replay via IPC. Result: p99 targets under 1 ms, per-node throughput ~700k–1M reqs/s, network egress drops roughly 5–6x, and incidents and buffering diminish. Replay core logic at edges works in production.
AI-written summary · about
chapters (9)
- 0:00Intro
- 1:05About us & our problems
- 5:50How can the system evolve safely & efficiently while performing?
- 9:45Benefits of determinism
- 20:12Can we optimize?
- 22:42Replay logic to scale & stabilize
- 31:1410 Challenges & consideration
- 38:02Simplicity
- 38:53Outro
related talks
20mFine Grained Replicated State Machines for a Cluster Storage System
44mThe Simplest Way to Build Resilient Applications
48mThe simplest way to build resilient applications
01h 09mAlgorithms for practical distributed agreement
45mDo Distributed Systems they said. It'll be fun they said.
35mBuilding resilient applications with Durable Execution
57mReliability Nirvana
34mDesigning, Building and Testing Deterministic HFT Systems