Finding Multiple Solutions of ODEs with Neural Networks
summary
Algorithm trains multiple neural nets with a new interaction loss that penalizes pairwise closeness so different networks converge to distinct roots; derivatives use automatic differentiation (TensorFlow/PyTorch style). Tests on two nonlinear initial-value ODEs and one boundary-value problem recover the known two and three solutions with MSE ≈ 1e-5 over 10k test points and a convergence loss threshold 1e-4. Uses simple two-layer 8-unit nets (sigmoid, skip connections), three-phase training that ramps interaction strength (k≈2) until solutions separate; fails for continuous families of solutions.
AI-written summary · about
chapters (16)
- 0:00Introduction to talk
- 1:05Why neural networks for ODEs
- 2:53Classical methods for ODEs
- 5:01Neural networks for solving ODEs
- 7:16Auto grad and differentiation
- 9:25Enforcing initial conditions
- 11:18Test equations overview
- 12:44Finding multiple solutions
- 15:20Training process phases
- 17:13Distance between solutions
- 18:14Full algorithm explained
- 20:26Role of lambda parameter
- 21:34Hyperparameter selection
- 24:15Architecture and results
- 27:43Conclusion and future work
- 29:11Q&A on infinite solutions
related talks
34mFast Algorithms for Multivariate ODEs arising in Sampling and Learning
01h 14mRecent Developments in Over-parametrized Neural Networks, Part I
49mAn Algorithmic Theory of Brain Networks
07mNeural Networks Demystified: Part 3
50mLearning and Generalization in Over-parametrized Neural Networks, Going Beyond Kernels
05mNeural Networks Demystified: Part 6
33mProvably Efficient Algorithms for Training Deep Networks
01h 51mUnderstanding Neural Networks (Somewhat)