Modernizing Ceph Deployments at CERN: CephFS and Object Storage Across Data Centres
summary
CERN runs Ceph at ~100 PBs across ~20 clusters in three locations and reports concrete moves: split huge file systems into multiple filesystems (multifs) collocated on the same hardware, integrate with OpenStack Manila and Kubernetes CSI, and use CBC/restic backups tied into Manila. Tests with MDTest (10 clients) show manual pinning and ephemeral distributed pinning improve metadata throughput versus ephemeral random or disabled dynamic balancer. Operational caveats: no client metadata throttling, crashes from pathological client patterns, DR needs explicit default-FS handling, and cross-DC S3 is manual failover.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:24CERN storage infrastructure
- 3:12CephFS at CERN today
- 5:19Daily CephFS challenges
- 7:05Modernizing CephFS deployments
- 8:11Dedicated file systems
- 10:40Availability zones in Kubernetes
- 11:46CephFS backups with CBEC
- 12:06Multiple file systems benefits
- 14:14Operating multifs clusters
- 15:20Disaster recovery in multifs
- 17:26Metadata balancing strategies
- 21:01Benchmarking metadata balancing
- 24:54Cross-data center CephFS
- 27:27Multi-site S3 object storage
- 29:14Future S3 storage plans
- 30:38Q&A session
related talks
49mIntro to Ceph, the Distributed Storage System
02h 59mCeph Storage: a Caffeinated Primer
35mMaking Ceph Fast in the Face of Failure
38mTesting Ceph: Pains & Pleasures
21mNew Techniques to Improve Small I/O Workloads in Distributed File Systems
29mGlobal Deduplication for Ceph
30mScaling Databases and File APIs with Programmable Ceph Object Storage
35mKeeping Ceph RGW Object Storage Consistent