Lifting the Curtain on Machine Learning
summary
Two R packages, polyreg and prVis, implement the core claim that many neural nets are effectively high‑degree polynomial regression: polyreg generates all degree‑D polynomial terms (usually D=2 or 3), fits them with lm/glm (no hyperparameters, unique solution), and outperforms Keras/deepnet on test sets (98.7% on EMNIST digits, ~80% on ForestCover, better MAE on Census and MillionSong). prVis feeds polynomial features into PCA and recovers the Swiss‑roll components more clearly than t‑SNE. Memory and matrix size are the main drawbacks.
AI-written summary · about
chapters (13)
- 0:00Introduction and context
- 1:14Critique of machine learning
- 3:05Neural networks explained
- 9:12History of neural networks
- 10:18Neural networks as polynomial regression
- 14:51Introducing polyreg package
- 21:18Implications of polynomial regression
- 24:33Testing on real datasets
- 31:58Image recognition limitations
- 35:11Nonlinear PCA with PRvis
- 39:20Team and backup slides
- 41:08Testing the theory
- 46:01Q&A session
related talks
59mTutorial: Statistical Learning Theory and Neural Networks I
01h 21mTutorial: Implicit Bias I
01h 06mAnalytic Methods for Supervised Learning III
14mScalable and Modular Robustness Analysis of Deep Neural Networks
56mHow neural networks learns simple functions?
23mparsnip A tidy model interface
32mStructured Deep Learning with Probabilistic Neural Prog
01h 12mPedagogical Talk: Frameworks for Information-Computation Tradeoffs