ShRing: Networking with Shared Receive Rings
summary
Multicore NIC receive rings default to 1024-entry rings with ~1.5 KiB per buffer, so per-core working sets reach ~1.5 MiB and the aggregate exceeds LLC/DDIO, causing high main‑memory traffic. ShRing shares a default-sized Rx ring among multiple cores, implemented on NVIDIA ConnectX‑5 (≈2,200 LOC) with Intel PCM, DPDK and 200 Gbps open‑loop tests. Sharing cuts memory bandwidth, raises NFV throughput up to 1.27×, and reduces latency up to 38× while falling back to private rings when a core is persistently slow.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:33Traditional sibling links and scalability
- 2:35Proposing ShRing to reduce working set size
- 3:59Link size impact on throughput and latency
- 5:04Why small links are impractical
- 5:46Sharing default-size links for scalability
- 6:49Polling vs packet processing with completion links
- 8:37Advancing head in shared links
- 10:02Pathological processing conditions
- 11:03ShRing on NVIDIA ConnectX-5
- 11:46Evaluating ShRing on load balancer
- 12:51Performance under pathological conditions
- 13:53Conclusion and key takeaways
related talks
18mDisentangling the Dual Role of NIC Receive Rings
14mODRP: On-Demand Remote Paging with Programmable RDMA
16mOneSidedMW: Managing Disaggregated Memory Efficiently, Flexibly, and Securely with RNIC
17mDRack: A CXL-Disaggregated Rack Architecture to Boost Inter-Rack Communication
16mRingLeader: Efficiently Offloading Intra-Server Orchestration to NICs
17mSRNIC: A Scalable Architecture for RDMA NICs
33mZero Copy Receive using io_uring
14mUnderstanding RDMA Microarchitecture Resources for Performance Isolation