Your Manila CephFS Share Backups Belong to S3
summary
CERN backs ~74 TB of production CephFS used by 65 projects and manages backups by snapshotting a PVC, creating a volume from that snapshot, mounting it and copying files to S3. The pipeline uses Velero with file-copy via Restic (peaks up to 25 GB memory, improved to ~8 GB in Velero 1.7.1), and is evaluating Kopia which halved elapsed time on a 1.5 million‑file kernel test and produced smaller S3 objects. CephFS CSI 3.7 adds constant‑time snapshot mounts but a snapshot‑name bug still blocks full automation.
AI-written summary · about
chapters (9)
- 0:00Intro
- 0:23Agenda
- 3:34Storage used at CERN Cloud
- 4:56Vocabulary
- 9:23Backup and restore workflow
- 18:11Backup & restoration solutions
- 25:49Where to go from here?
- 27:12Restic and Kopia comparison
- 29:19Conclusion
related talks
29mCeph Object Storage Overview, Capabilities and Future Plans
47mRunning Backups with Ceph-to-Ceph
43mCeph and Flash
31mAI, ML, and the Ceph Advantage: Scalable Storage for Smarter Workflows
33mThe Challenge of Storing Small Objects on a Large Scale
16mProxmox Backup Server: Backup for your Datacenter
14mFalconFS: Distributed File System for Large-Scale Deep Learning Pipeline
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...