Scalable Small Message Aggregation on Modern Interconnects
summary
Proposes aggregation contexts as an OpenSHMEM extension that adds one context argument and a quiet flush to defer small remote operations, encoding metadata into fixed-size packets and using conveyors underneath. Validates at scale and across platforms — Frontier (Slingshot), Andes and Iris (InfiniBand) — up to 8192 nodes, showing up to 65× speedup on InfiniBand and ~15× on Slingshot, near-peak packet efficiency at 64 PPN, and sensitivity to shared-job interference on large runs.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 2:42Aggregation strategies
- 4:28OpenMPI extension design
- 7:40Histogram benchmark
- 9:08Frontier setup
- 11:16Scalability results
- 13:26Message rate performance
- 14:30Cross-architecture comparison
- 15:548000-node test
- 17:43Scheduling impact
- 18:45Aggregation efficiency
- 19:31Summary
- 20:38Future work
- 20:59Q&A
related talks
21mFuture Challenges for HPC Networking @Scale
11mScaling Distributed Machine Learning with In-Network Aggregation
25mShoal: A Network Architecture for Disaggregated Racks
33mSUPERp: Scale UP EtherRnet Protocol
19mDisaggregating Stateful Network Functions
17mNetHint: White-Box Networking for Multi-Tenant Data Centers
16mRingLeader: Efficiently Offloading Intra-Server Orchestration to NICs
24mConfluo: Distributed Monitoring and Diagnosis Stack for High-speed Networks