Partial Evaluation of Forward-Mode Automatic Differentiation

on YouTube ↗05m

" summary

A whole-program transformation called PEAD (partial evaluation automatic differentiation) combines compile-time symbolic differentiation with forward-mode AD to generate code that eliminates much dual-number overhead by partially lifting only the differentiation-affected code and then partially evaluating the AD-generated dual operations. The pipeline explicitly handles lifting and differentiation of free variables, uses the type system to keep both normal and lifted versions of functions to avoid blind code explosion, and enforces a user-defined max recursion unroll to pragmatically bound partial-evaluation expansion; produces significantly leaner runtime code for many scalar cases.

AI-written summary · about

" chapters (3)

1. 0:00Partial Evaluation of Forward-Mode Automatic Differentiation
2. 0:53Research Challenge
3. 1:21Approach Overview

" related talks

• 29mPartial Evaluation of Automatic Differentiation for Differential-Algebraic Equati...David Broman, Viktor Palmkvist, Oscar Eriksson · GPCE 2023
• 26mAdding AD to Scheme by Differentiating the InterpreterBarak A. Pearlmutter, Mehrdad Maleki · Scheme 2021
• 11mAbstract Interpretation for Automatic DifferentiationGagandeep Singh, Jacob Laurel, Sasa Misailovic, Siyuan Brant Qian · LAFI 2024
• 30mAutomatic Differentiation in PCFDamiano Mazza, Michele Pagani · POPL 2021
• 36mProgramming Languages for Automatic Differentiation: What Now?Damiano Mazza · LAFI 2022
• 59mDifferentiable Programming in C++Vassil Vassilev, William Moses · CppCon 2021
• 26mADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Progra...Vikash K. Mansinghka, Mathieu Huot, Alexander K. Lew, Sam Staton · POPL 2023
• 29mParallelism-preserving automatic differentiation for second-order array languagesadam basker · FHPNC 2021