Higher order graph neural networks with P-tensors
summary
Introduces P-tensors as a formalism that enforces permutation equivariance for higher‑order message passing and extends to Schur‑nets to expose local automorphism structure; counts of linear equivariant maps match combinatorics (matrices → 15 maps, general case → Bell number for partitions), implements custom CUDA kernels and a PyTorch‑style library to enumerate einsum‑style ops for overlapping subgraph tensors, demonstrates practical correctness and speedups but finds batching, CPU precomputation and multi‑GPU scaling remain the main engineering bottlenecks.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:02Graph neural networks beyond GNNs
- 8:56Permutation equivariance
- 21:10Equivariant linear maps
- 28:04Permutation equivariance limits
- 34:05Message-passing GNN shortcomings
- 49:21Higher-order GNNs with P-tensors
- 1:02:58Future directions
- 1:04:24Q&A
related talks
34mThe physics of deep learning using tensor networks
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
15mPET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated
10mJigsaw: Domain and Tensor Parallelism for High-Resolution Inp...
47mPyTorch Geometric for Graph Neural Nets
19mNeuGraph: Parallel Deep Neural Network Computation on Large Graphs
10mDale Smith, Data Scientist, Nexidia
58mIntroduction to the tensor-programs framework, a PL approach that helps analyse ...