Exploring the Performance Limits of CephFS in Nautilus
summary
Benchmarking Nautilus CephFS with FIO, smallfile and a harness called dimmer shows clear limits: NVMe drives deliver ~400k 4K random IOPS and ~2–5 GB/s sequential, while a single CephFS mount on 25GbE hits roughly half of that for reads and about 75% for sequential writes. Random I/O falls to ~14–15k IOPS from a three-OSD PG, and small-file create throughput tops out near 125–130 MB/s per active MDS (adding a second MDS roughly doubles throughput). Kernel client caches outperform FUSE.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:16Background
- 2:10Areas of Improvement
- 3:43Tools
- 5:59Hardware Configuration
- 6:16Hardware Hardness
- 7:16Sequential Performance
- 8:00Small File Data Sets
- 17:27Random Right Performance
- 18:08Small File Performance
- 21:00Latency
- 21:21Concurrency
- 22:41Stats
- 24:18Increasing the number of MDS
- 25:00Maximum throughput
- 26:11Cache effectiveness
- 28:53Next steps