Empowering Azure Storage with RDMA
summary
Azure replaces TCP with RoCEv2-based RDMA at regional scale, moving roughly 70% of traffic to RDMA and handling exabytes per day over 100+ Gbps links up to ~100 km. The deployment adds RDMA e‑stats for fine-grained latency and synchronized NIC/CPU timestamps, an automatic TCP fallback for lossy links, and tuned storage libraries that translate socket APIs to RDMA verbs. Key protocols and fixes include PFC tuning, ECN-based congestion control like DCQCN, extensive testing, and interoperability work across diverse NICs.
AI-written summary · about
chapters (14)
- 0:00Introduction to Azure RDMA
- 1:32The problem with TCP
- 3:13Importance and benefits
- 4:22The RDMA solution
- 5:57RDMA evolution and RoCE
- 7:24Challenges of implementation
- 8:41Enabling RoCE at scale
- 11:08Congestion and blocking
- 12:27Congestion control protocols
- 14:29Software and driver support
- 16:09Performance monitoring
- 17:52Lessons and best practices
- 19:14Open research problems
- 19:41Conclusion
related talks
25mRDMA @Scale
15mSecuring RDMA for High-Performance Datacenter Storage Systems
12mPreview: Networking
20mRDMA at Cloud Scale: The OCI Experience
18mFileMR: Rethinking RDMA Networking for Scalable Persistent Memory
18mRoCE BALBOA: Service-Enhanced RDMA Offload Engine for Data Center SmartNICs
22mMetaRoCE: Meta’s RDMA Transport
18mRDMA is Turing complete, we just did not know it yet!
