Stretch Clusters in Ceph: Algorithms, Use Cases, and Improvements
summary
Implements a two‑site stretch cluster mode for Ceph that enforces two copies per site, restricts OSDs to local monitors, and adds a tiebreaker monitor; replaces lowest‑ID leader selection with leader scoring (gossiped peer reliability) and unit‑tested election logic; changes OSD peering via a new calc_replicated_acting_stretch path and requires two monitors per site. Works in production for a few users but omits erasure coding and three‑site support, revealed Rook‑driven monitor ID bugs, and relied on Red Hat QA to catch edge cases.
AI-written summary · about
chapters (9)
- 0:00Introduction to stretch mode
- 1:47Stretch cluster challenges
- 4:32Monitor election mechanics
- 7:20Implementing the new algorithm
- 12:12Peering and data recovery
- 16:05System-wide implementation
- 18:48Bugs and development lessons
- 21:48Limitations and future work
- 25:13Setup and Q&A
related talks
25mExplicitly Supporting Stretch Clusters in Ceph
25mSupporting 3 Availability Zones Stretch Cluster
27mDetect, Decide, Fence: Resolving Netsplits in 3-AZ Stretch Cluster
27mFrom Chaos To Clarity: Systematic Ceph Cluster Analysis for Rapid Troublesho...
49mIntro to Ceph, the Distributed Storage System
34mModernizing Ceph Deployments at CERN: CephFS and Object Storage Across Data Centres
29mCeph Operations at Scale
02h 59mCeph Storage: a Caffeinated Primer