MGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...
summary
MGG is a full-graph GNN runtime that implements a fine-grained dynamic software pipeline inside GPU kernels to overlap remote-label fetch, local neighbor loads, and aggregation, plus an analytical runtime that partitions labels, interleaves local/remote work, and performs GPU-aware pipeline mapping to stream processors. Implemented and measured on an NVIDIA DGX A100, it reports average speedups of 4.41× vs DGL, 4.81× vs MGG‑UVM and 10.83× vs ROC, and shows larger gains on very sparse, large graphs (examples up to 100M nodes / 1.6B edges).
AI-written summary · about
chapters (9)
- 0:00MGG project intro
- 2:39Multi-GPU GNN training challenges
- 6:12Fine-grained pipelining for GNNs
- 8:40MGG design and runtime
- 10:27Pipeline workload management
- 11:50GPU-aware pipeline mapping
- 13:13Performance evaluation
- 14:39Contributions summary
- 15:00Q&A
related talks
13mGNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs
17mBGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
16mAccelerating Distributed Graph Learning by Using Collaborative In-Network...
19mNeuGraph: Parallel Deep Neural Network Computation on Large Graphs
29mGraph Neural Networks + LLM on
16mGMI-DRL: Empowering Multi-GPU DRL with Adaptive-Grained Parallelism
15mIdentifying and Analyzing Pitfalls in GNN Systems