Demystifying Hyper-Dense Ceph: How To Design for Large Capacity NVMe
summary
Claims that hyper‑dense Ceph with >30TB NVMe can match availability if layout and device settings match real device characteristics, supported by concrete changes: a PR to auto‑query optimal_io_size check and set bluestore min_alloc at OSD deployment, lsblk/qemu and vendor SMART fields for verification, and a Solidigm tool called endurance amplifier for write‑amplification metrics. Measured W jumps from ~1 to 1.6 in two hours when misaligned; recommendations include more EC fragments, fewer drives per node, and accepting longer rebuilds.
AI-written summary · about
chapters (11)
- 0:00Introduction and motivation
- 1:03High-capacity NVMe drives overview
- 3:14Ceph cluster size limits
- 5:47Drive endurance and PBW explained
- 7:38Indirection units and alignment
- 11:18BlueStore and indirection units
- 15:16BlueStore min_size and alignment
- 19:10PR for automatic alignment
- 23:10Design trade-offs for large drives
- 25:20Erasure coding strategies
- 27:53Q&A