CephFS as a Scalable Filer
summary
Monash runs CephFS as a production filer on ~100 servers, ~2000 OSDs with two data pools (216 and 342 OSDs), a carved SSD metadata pool, five front-end VMs and AD auth, backed to TSM/tape. CephFS outperforms the prior RBD+ZFS virtual filers for streaming (roughly double bandwidth and lower latency) but shows large slowdowns on metadata-heavy workloads (ls, create/unlink of 10k–100k small files often 3–5× slower). Best results come from CephFS via kernel mount rather than FUSE; exports use Ganesha NFS and Samba over FUSE in current deployment.
AI-written summary · about
chapters (12)
- 0:00Introduction to Monash and CephFS
- 1:10Monash University overview
- 4:01Research infrastructure at Monash
- 8:23Why choose CephFS at Monash
- 10:54CephFS-backed filer architecture
- 15:14Performance comparison overview
- 17:23NFS performance results
- 21:20SMB performance results
- 24:43Kernel vs FUSE mount comparisons
- 29:34NFS vs CephFS native performance
- 33:46Key takeaways from performance tests
- 36:36Q&A session