Apex
summary
Apex is an open-source PyTorch extension that uses mixed precision (FP16 for compute, FP32 for accumulators) to exploit NVIDIA tensor cores and deliver large speedups—bandwidth-limited ops get ~2x, matrix multiply/convolution on tensor cores shows up to 8x throughput. Automatic mixed precision (AMP) requires only a few lines of Python to cast operations appropriately, preserves final accuracy comparable to FP32 without retuning, reduces memory by ~2x, and is being upstreamed into PyTorch for native support.
AI-written summary · about
chapters (5)
related talks
10mPyTorch Performance
12mMixPert: Optimizing Mixed-Precision Floating-Point Emulation on GPU Integer Tensor Cores
25mmxfp8, mxfp4, nvfp4 formats and applications in PyTorch
22mCampo: Cost-Aware Performance Optimization for Mixed-Precision Neural Network...
25mPyTorch APIs for High Performance MoE Training and Inference
25mAI acceleration in POWER10
25mFp8 Training From Hopper To Blackwell
26mEfficient MoE Pre-training at Scale on AMD GPUs With TorchTitan