Best Practices of Production-Grade Rook/Ceph Cluster
summary
On-prem Kubernetes runs Rook managing two Ceph clusters: RBD on NVMe + RGW on HDD with NVMe used for indexes and HDD for bulk data, OSDs auto-created and spread using topologySpreadConstraints so OSDs land evenly across racks and hosts. NVMe OSDs use dynamic provisioning via topoLVM while HDD PVs are preprovisioned. A homemade script replaces dead OSDs without reducing replica counts. Daily upstream monitoring catches bugs; upgrade to Ceph 17.2.6 patches an RGW data loss bug. custom containers are used for emergency fixes and test automation.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:03Kubernetes and Ceph basics
- 2:30Why Rook/Ceph at Cybozu
- 4:39Cluster architecture overview
- 8:15Rook operator fundamentals
- 10:03Advanced Rook cluster setup
- 15:15Cluster operations
- 18:15Troubleshooting Ceph
- 19:00OSD deployment strategies
- 23:00Automatic OSD provisioning
- 24:50SSD and HDD configurations
- 26:42Daily upstream checks
- 28:28Community contributions
- 29:53Custom container testing
- 31:19Future work and automation
- 32:04Conclusion and Q&A
related talks
19mUsing Rook in Production on National Research Platform
29mSimplifying Storage in Kubernetes with Rook-Ceph
31mRook: Intro and Deep Dive With Ceph
01h 21mHands On with Rook: Ceph & Kubernetes
35mDeploy Ceph in Kubernetes: Rook Deep Dive
41mRook - Running Ceph Using Kubernetes
31mRook: Intro and Deep Dive With Ceph Storage
27mRook: Intro and Ceph Deep Dive