Automatic Differentiation via Effects and Handlers in OCaml
summary
Implements four AD algorithms in OCaml 5.0 using effects and handlers to bubble up arithmetic and capture continuations, with a reverse-mode functor that pairs values with mutable accumulators and a grad helper for gradients and nested (higher‑order) derivatives. Extends to tensor ops via Owl (35 tensor/scalar primitives) and runs MLBench GMM tests (N=1k, 10k, KD varied). Microbenchmarks show correct O(n) scaling and about a 8× slowdown over raw evaluation; macro results are competitive with dynamic systems, second only to PyTorch/TensorFlow*.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:22Automatic differentiation
- 1:25Forward/reverse chain rule
- 5:14Effects and handlers
- 7:24Smooth effect
- 10:31Floating-point operations
- 13:24Reverse mode with handlers
- 16:29Computing derivatives with grad
- 18:37Reverse mode performance
- 19:18Micro benchmarks
- 20:41Macro benchmarks
- 24:55Conclusion
- 26:37Q&A
related talks
01h 09mLatent Effects and Handlers + Automatic Differentiation via Effects and Handlers
29mPartial Evaluation of Automatic Differentiation for Differential-Algebraic Equati...
36mProgramming Languages for Automatic Differentiation: What Now?
29mReverse Automatic Differentiation for Accelerate (Extended Abstract)
17mMachine Learning Primitives as Algebraic Effects
47mIntroduction to Algebraic Effects
11mAbstract Interpretation for Automatic Differentiation
35mDerivatives - Important Concept. Simple to grasp in Kotlin