Array Programming on GPUs: Challenges and Opportunities
summary
Binary instrumentation at the SASS/PEX level finds pervasive floating-point exceptions across 256 GPU ports and pinpoints kernels that produce NaNs or infinities; an example training run that finishes in 2s uninstrumented becomes 4 minutes under the analyzer, while autograd anomaly detection costs ~3x slowdown. The extended tool GPU-FPX captures per-lane exception flags for Tensor Cores, and targeted tests FTTN reveal blockwise FMA widths that make an 8K×8K D=C−AB produce 255 on an MI100 and 191 on an H100 instead of 0. An SMT model of multiple Tensor Core generations enables formal checks and helps explain cross-device variability.
AI-written summary · about
chapters (8)
- 0:00GPU array programming intro
- 1:23GPU hardware overview
- 3:12Energy efficiency in HPC
- 7:28Floating-point exceptions impact
- 14:09Exception detection tools
- 19:50Hardware specs variability
- 23:25Formal specs and verification
- 25:51Summary and future directions
related talks
01h 06mFoundation of GPU Programming
12mMixPert: Optimizing Mixed-Precision Floating-Point Emulation on GPU Integer Tensor Cores
53mIntroduction to C++ AMP (GPGPU Computing)
01h 18mOptimizing LLM Training and Inference Performance on GPUs
19mOptimizing for GPUs
08mGPU Programming with Triton Kernels
12mContinuous Profiling for GPUs
28mAuto-instrumentation for GPU Performance Using eBPF