Compiling to Recurrent Neurons
summary
Defines a linearly typed higher-order language Cajal(N) that compiles iteration and conditionals into matrices realized as recurrent neurons, proves a compiler correctness theorem showing compiled programs preserve semantics, and ships artifacts. Experiments embed compiled circuits into a network to solve an image task (apply blur k times when input digit is k if odd, identity if even), and the compiled models train faster and with markedly better data efficiency than naive baselines, converging within the first few hundred to ten thousand steps.
AI-written summary · about
chapters (10)
- 0:00Programming languages for neural nets
- 1:24History of high-level languages
- 3:50Limitations of neural networks
- 5:33Introducing Kahal calculus
- 7:58Compiling to differentiable matrices
- 9:23Compiler correctness and integration
- 10:48Experiment: discrete algorithms to networks
- 13:14Scaling compiled algorithms
- 14:40Future directions
- 15:00Q&A
related talks
23mCompiling to Linear Neurons
13mTowards Compiling Higher-Order Programs to Bayesian Networks
55mComputation with Assemblies
12mComputation with sequences of neuronal assemblies
37mNeural Networks are Programs, Too
37mCode First, Math Later: Learning Neural Nets Through Implementation and Examples
01h 07mTheory of Computation II
43mCoin-Flipping in the Brain: Probabilistic Computation and Learning in the Assembly Model