Compiling to Linear Neurons
summary
Claims a typed linear language called Cajal compiles discrete programs into linear neurons by translating conditionals into soft branching linear maps, with a formal correctness theorem and a crucial lemma tying linear types to linear algebra. Implements a 200-line PyTorch extension, trains on a conditional image-to-one task for up to 3,000 steps, evaluates PSNR across 5 seeds and varied hyperparameters, and shows faster, more data-efficient learning and better identity reconstruction than indirect baselines.
AI-written summary · about
chapters (16)
- 0:00Compiling to linear neurons
- 1:02Conditional image transformation
- 3:29Direct vs indirect programming
- 4:33Kahal calculus
- 5:15Linear neurons vs linear maps
- 6:38Linear typing in compilation
- 7:45Hard branching problem
- 9:09Soft branching solution
- 10:13Compilation rules and correctness
- 11:38Testing image task correctness
- 12:21Direct learning dynamics
- 13:48Indirect learning dynamics
- 15:11Model comparison
- 16:14Extending Kahal
- 16:55Differentiable programming thoughts
- 17:59Question and answer session
related talks
18mCompiling to Recurrent Neurons
37mNeural Networks are Programs, Too
39mLinear Logic Programming
32mStructured Deep Learning with Probabilistic Neural Prog
01h 00mBetter Learning through Programming Languages
18mActive Learning for Neurosymbolic Program Synthesis
12mLanguage-Model Probabilistic Programming for Improving Autoformalization via Cycle(…)
01h 07mTheory of Computation II