Immutable Values in the Big Wide World
summary
Kevel replaces a global Redis replication tree with S3-backed value change streams that store immutable versioned values and reified deltas, using LMDB/memory-mapped checkpoints and roaring-bitset in‑RAM fan-out to compute visibility. The system uses hybrid logical clocks to provide causal+ consistency and coordination-free read transactions across refs. Fan-out compute drops from about 400 machines to three partitions in practice, single-node update load reaches ~350k req/s, and the design enables time‑travel debugging and fast cross‑region recovery.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 5:252018 system architecture
- 16:18Legacy system problems
- 22:24Functional programming and distributed systems
- 25:55Value Change Streams explained
- 34:02New system evolution
- 37:12Current architecture with VCS
- 40:44Key benefits of new model
- 46:03Future goals and impact
related talks
13mDifferentiated Key-Value Storage Management for Balanced I/O Performance
11mEvolution of Development Priorities in Key-value Stores Serving Large-scale Applications
15mKVIMR: Key-Value Store Aware Data Management Middleware for Interlaced Magnetic...
33mDiscover Valkey: The Path to Now
37mScaling Vinted: A System That Never Stands Still
05mExploring Data Deduplication in LSM Tree-based Key-Value Stores (Work-in-Progress Report)
31mReimagining the Future: Where the Market is Evolving
18mSplitKV: Splitting IO Paths for Different Sized Key-Value Items with Advanced