How Rook/Ceph Enables Science @ Rubin Observatory
summary
Rubin Observatory runs a consolidated Rook/Ceph stack that handles ~20 TB of raw images per night, transforms 2–3 GB focal-plane exposures, and pushes data within 7 seconds to two RGW endpoints (summit and USDF). Current on-site Rook clusters use OSD encryption and a mix of HDD and NVMe, with a largest planned cluster exceeding 1100 OSDs and the summit RGW sized at about 3 PB for 30 nights. Operations use GitOps (Flux/Argo/Fleet, Renovate) and keep low-latency workloads on local NVMe.
AI-written summary · about
chapters (7)
- 0:00Introduction and project overview
- 3:57Rubin Observatory’s data flow
- 8:26Why choose Rook and Ceph
- 12:16Consolidating storage with Rook
- 15:45Current Rook cluster fleet
- 16:50GitOps for storage provisioning
- 21:39Q&A on infrastructure and upgrades
related talks
31mRook: Intro and Deep Dive With Ceph Storage
31mRook: Intro and Deep Dive With Ceph
29mSimplifying Storage in Kubernetes with Rook-Ceph
27mRook: Intro and Ceph Deep Dive
26mRook: Intro And Deep Dive With Ceph Storage
35mBest Practices of Production-Grade Rook/Ceph Cluster
12mRook: Why Would You Ever Deploy Ceph Inside Kubernetes?
30mRook: Intro and Ceph Deep Dive