NVMeoF in Ceph - What's New and Performance Best Practices
summary
Ceph’s Tentacle release ships NVMeoF with multi-gateway Tentacle gateway groups that store config in a shared OMAP and use a monitor with Paxos-style beacons for HA, plus ANA groups that make each namespace active‑passive per gateway while allowing parallel access. Tested scale reaches 32 gateways (4 groups), 16k namespaces, and a configured best case of 1 million IOPS under 2 ms with 12 gateways; targets include 4–6 second failover, per-namespace QoS, namespace masking (8 hosts), SPDK cancel contribution, and a CSI driver tech preview.
AI-written summary · about
chapters (21)
- 0:00Introduction and presenters
- 1:23Tentacle release and NVMeoF support
- 3:10High availability with gateway groups
- 8:34Security and namespace masking
- 10:45CLI integration and NVMe reservations
- 12:33NVMe cancel command and QoS
- 15:05New features and Kubernetes CSI
- 17:09NVMe service improvements
- 18:54Stretch cluster optimization
- 19:56Failover time reduction
- 20:37CRC optimization research
- 21:18Performance optimization efforts
- 21:41Performance test setup
- 23:30OSD density and SPDK cores
- 24:14Q&A session
- 28:14NVMeoF vs RBD compatibility
- 28:37Performance comparison with RBD
- 29:23DPU and SmartNIC testing
- 30:27Namespace usage in production
- 32:35Performance test details
- 33:40Closing remarks
related talks
38mCeph NVMe-of Road Map
44mNVMe-Over-Fabrics Support for Ceph
30mFrom ISCSI to NVMe-of: A Comparative Look on Storage Protocol Advantages & Challenges
46mUnified Cluster Federation: A Hyperscaled Architecture of...
18mCStar Gateway: Augmenting Public Cloud Infrastructure for Heterogeneous Network...
49mPetabyte CephFS for Satellite Data Processing
14mCETOFS: A High-Performance File System with Host-Server Collaboration for Remote Storage
26mApplying New High-Throughput Network Solutions in IEC (Integrated Edge Cloud)