Revealing Floating-Point Accumulation Orders in Software/Hardware Implementations
summary
A diagnostic tool called FPRev nonintrusively infers the actual floating point accumulation tree by feeding specially crafted inputs that exploit precision swamping and observing outputs, reducing exponential brute force testing to polynomial calls and further pruning redundancy to near the lower bound of required tests. FPRev reveals differing orders such as NumPy doing two‑way accumulation on some CPUs versus sequential on others, and exposes variations across NumPy and PyTorch on CPUs and GPUs including Tensor Cores; the tool is open sourced.
AI-written summary · about
chapters (11)
- 0:00Floating-point accumulation orders
- 1:48Numerical reproducibility challenges
- 3:55Accumulation order impact
- 5:23Determining accumulation orders
- 6:04FPRev diagnostic tool
- 8:31Reconstructing accumulation orders
- 12:03Optimizing FPRev’s algorithm
- 13:49Efficient summation trees
- 15:59Experimental setup
- 17:23Performance results
- 18:47Summary
related talks
01h 00mFastware
23mBeating the Benchmarks: Co-evolving the ISA and Development Tools
44mDid You Know What You Just Counted? Automated Verification of Numerical Computations
50mRecent Advances in Floating-point (Static) Analyses
15mCost of Soundness in Mixed-Precision Tuning
22mModular Optimization-Based Roundoff Error Analysis of Floating-Point Programs
14mOracle-Free Repair Synthesis for Floating-Point Programs
19mHow NOT To Program an Out-of-order Vector Processor