Using Rook in Production on National Research Platform
summary
Experience running 8 Ceph clusters under Rook inside one Kubernetes cluster with a multi-petabyte footprint: 13 PB raw / 800 OSDs and about 12,000 CSI volumes. Rook’s full integration auto-installs CSI and exposes Ceph via CRDs so teams avoid manual Ceph setup, but operator behavior creates single-failure blast radius, upgrades block when one cluster is broken, overlay networking cripples performance so host network preferred over overlay, and NVMe DBs require one large NVMe per node.
AI-written summary · about
chapters (8)
- 0:00Introduction to NRP
- 5:24Rook deployment overview
- 8:21SE pools in NRP
- 10:10Benefits of Rook
- 12:21External cluster mode
- 14:06Host path volumes
- 15:11Challenges with Rook
- 18:41Conclusion and Q&A
related talks
31mRook: Intro and Deep Dive With Ceph Storage
29mSimplifying Storage in Kubernetes with Rook-Ceph
12mRook: Why Would You Ever Deploy Ceph Inside Kubernetes?
31mRook: Intro and Deep Dive With Ceph
41mRook - Running Ceph Using Kubernetes
26mRook: Intro And Deep Dive With Ceph Storage
07mRook: Enterprise Storage for Kubernetes
55mIntroducing Rook