(De/Re)-Composition of Data-Parallel Computations via Multi-Dimensional(…)
summary
Formal algebraic framework Multi-Dimensional Homomorphisms expresses data-parallel kernels with exactly three higher-order patterns (infu mdom outw), a low-level decomposition/recomposition pipeline, and performance parameters that drive automatic space exploration via ATF autotuning. Implementation generates OpenMP/CUDA code and auto-tunes blocks; experiments on BLAS, stencils, quantum chemistry, data-mining and deep-learning workloads show speedups >2x vs TVM and up to >9x vs vendor libs (cuBLAS/cuDNN, oneMKL/oneDNN) on real inputs, with an nvrtc backend causing some noted slowdowns.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:02Research questions
- 2:47Major contributions
- 3:49Representation goals
- 5:34Data-parallel with three functions
- 7:19Hardware-optimized decomposition
- 8:21Visualizing optimizations
- 9:05Lowering process
- 9:47Experimental setup
- 10:50Deep learning results
- 11:31PEN metric evaluation
- 12:13Productivity comparison
- 12:54Conclusion
- 13:16Q&A
related talks
17mData-Parallel Differentiation by Optic Composition
30mData-Parallel Flattening by Expansion
01h 07mGeneric Parallel Programming
20mExploiting Sparsity and Redundancy
01h 02mMulti-Dimension Mapping of Embedded Applications
17mAccelerating Collective Communication in Data Parallel Training across Deep Learning...
18mCompiling Strassen-like Matrix Multiplication Algorithms to Fast CUDA Kernels
30mGenerating High Performance Code for Irregular Data Structures using Dependent Types