Disaster Recovery of Stateful Applications in a Multi-Cluster Environment

on YouTube ↗35m

" summary

Demonstrates a concrete DR system built on Ceph/Rook + RBD and a CSI extension (VolumeReplication) with Open Cluster Management to align clusters and Ramen DR orchestrator to run failover and relocate actions. Demo shows failover losing about one minute of writes (RPO ≈ 1 min) with recover time ≈ 1m20s, and relocate achieving RPO=0 with ~1m30s downtime. Operators run on a hub and managed clusters, auto-create replication relationships, and reconcile PV/PVC mappings for dynamic provisioning.

AI-written summary · about

" chapters (25)

1. 0:00Intro
2. 0:56What is Ceph?
3. 1:38What is Rook?
4. 2:06Why Disaster Recovery?
5. 3:12Disaster Recovery and High Availability
6. 3:35RPO and RTO
7. 4:05An Ideal DR Solution
8. 4:36Example: Rook+Ceph Regional DR
9. 6:00Regional DR Goal - Relocate
10. 6:49Backups as Replication
11. 9:03Storage System Based Replication
12. 10:05Storage Replication Management APIs
13. 11:56Recovery Management
14. 13:20Dynamic Provisioning Conundrum
15. 15:23Cross Cluster Actions
16. 17:00Open Cluster Management - 101
17. 18:28Ensuing User Complexity: Deploy
18. 19:20Ensuing User Complexity: Recover
19. 20:37Ensuing User Complexity: Relocate
20. 21:34DROrchestrator: Ramen
21. 22:29Ramen: DRPolicy API
22. 23:10Ramen: DRPlacementControl API
23. 24:24Ramen: Operator Deployments
24. 31:29Future work and adaptations
25. 34:15Links and References

" related talks

• 46mKubernetes Stateful Application Disaster Recovery with Zero Data LossDaniel Parkes · Cephalocon 2023
• 30mWhatever Can Go Wrong, Will Go Wrong – Rook/Ceph and Storage FailuresSagy Volkov · KubeCon + CloudNativeCon North America 2020 Virtual
• 30mIs Your Kubernetes Disaster Recovery Actually Ready?Saiyam Pathak, Saloni Narang · KubeCon + CloudNativeCon Japan 2026
• 25mDynamic Workloads need Dynamic Storage - using rook-ceph with k8sSteven Ellis · linux.conf.au 2020
• 35mBest Practices of Production-Grade Rook/Ceph ClusterSatoru Takeuchi · Cephalocon 2023
• 15mWhen Clusters Collide: The Misadventures of Disaster Recovery in K8sLev Andelman · Conf42 Platform Engineering 2024
• 29mTowards Sustainable Multi-Cluster Management in Real-World Academic InfrastructureJ. Mok, C. Im · KubeCon + CloudNativeCon Japan 2026
• 31mRook: Intro and Deep Dive With CephSatoru Takeuchi, Dan van der Ster · KubeCon + CloudNativeCon Japan 2026