NVMe-Over-Fabrics Support for Ceph
summary
A NVMe-over-Fabrics Gateway for Ceph translates NVMe/TCP and RDMA to RADOS/RBD using an SPDK target and a Python gRPC control daemon. Configuration persists in Ceph omap to support multi-gateway groups and multipathing. Added RBD operations: unmap, flush, write-zero and larger compare-and-write (compare emulated in SPDK); abort is best-effort and copy is discouraged. Single-Gateway tests reach 74–85 Gbit on 100Gb with 400 volumes using SPDK target and a Python control daemon. Repo is upstream but not production-ready.
AI-written summary · about
chapters (13)
- 0:00Introduction and motivation
- 1:52NVMe-over-Fabrics Gateway overview
- 3:17Subsystem and namespace mapping
- 5:04SPDK data path design
- 6:29NVMe command to RBD mapping
- 11:09Multi-gateway groups and failover
- 16:10Performance benchmarks
- 20:02Future plans and Discovery service
- 23:15Identification and encryption
- 25:01Quality of Service support
- 27:09Direct OSD NVMe-over-Fabrics challenges
- 29:40Current status and roadmap
- 31:48Q&A session
related talks
34mNVMeoF in Ceph - What's New and Performance Best Practices
38mCeph NVMe-of Road Map
30mFrom ISCSI to NVMe-of: A Comparative Look on Storage Protocol Advantages & Challenges
37mSPDK Based User Space NVMe Over TCP Transport Solution
37mIntroducing Ceph Mgmt-gateway: Unified Access, OIDC-based SSO, and HA for Manageme...
17mWhite-Boxing RDMA with Packet-Granular Software Control
39mSecured I/O in the Cloud with NVMe-Over-TCP
17mCo-Designing Traffic Control with NVMe-oF for Disaggregated Storage: A Comparative Study