Demystifying Valkey Clustering: Architecture, Fault Tolerance, and Scalability
summary
Valkey Cluster is a mesh of TCP connections with gossip-based membership and slot sharding using CRC16 mod 16384; one primary can own many slots and clients get redirected. Failures use ping/pong with a 15s node timeout, quorumed failover via monotonically increasing epochs, and slot migration; benchmarks show 2,000 nodes (666 primaries, two replicas per shard) with 10% failures converging in ~20s, serialized extremes ≤60s, ~2% CPU and ~2 Mbps network overhead, and a new 30 byte light header for pubsub.
AI-written summary · about
chapters (11)
- 0:00Introduction to Valkey
- 1:04Valkey operational modes
- 3:13Key components of clustering
- 10:01Node membership and redirection
- 13:31Failure detection mechanism
- 18:50Failover process explained
- 22:02Epoch values and conflict resolution
- 26:39Scalability benchmarks
- 29:10Recent clustering improvements
- 33:05Future enhancements and challenges
- 35:59Q&A session
