Reverse mode ADEV via YOLO: tangent estimators transpose to gradient estimators
summary
Claims an automated pipeline that converts ADEV's forward-mode tangent estimators into a reverse-mode estimator by applying YOLO’s unzip and transposition steps to the forward-generated Q program. Implements ADEV in JAX, runs YOLO transforms on the estimator programs, and trains variational autoencoders with efficiency rivalling best-in-class variational-inference engines. Handles programs with sampling and parameter-dependent expectations to produce unbiased gradient estimators, avoids repeated forward invocations, and for a JAX-equivalent subset the transformation is compile-time only.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:46Standard AD limits
- 3:10Gradient estimation strategy
- 4:59POLE solution AD
- 6:42Reverse mode via YOLO
- 8:51Empirical results
- 9:56Summary and future
related talks
26mADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Progra...
25mYou Only Linearize Once: Tangents Transpose to Gradients
21mProbabilistic Programming with Programmable Variational Inference
46mGradient Descent: The Ultimate Optimizer
11mIntro to JAX: Accelerating Machine Learning research
14mJAX: Accelerated Machine Learning Research via Composable Function Transformations in Python
42mRunning Programs In Reverse for Deeper A.I.
29mReverse Automatic Differentiation for Accelerate (Extended Abstract)