The mathematics of machine learning
summary
Claim: the target conditional expectation is (for practical purposes) a continuous function on the unit hypercube, so any family that densely spans continuous functions — indicator/step functions, multivariate polynomials, piecewise linears, trig polynomials, neural nets — can approximate it to arbitrary accuracy. The protocol is: pick an intuitive function family, choose an error metric that matches a convergence mode (Lp, prob, a.s.), grow complexity incrementally, favor parsimony, and prove convergence. Empirically supplying analytic derivatives cuts runtime by ~2/3–3/4; one case yields 0.76 vs 0.77 Gini for 151 vs 10,000 parameters while recovering an interpretable inflation signal — explicable precision.
AI-written summary · about
chapters (16)
- 0:00Introduction and goals
- 0:44The role of predictive modeling
- 3:26The need for explicability
- 6:05Current state of machine learning
- 9:22Critique of data science practices
- 13:12A mathematical framework
- 16:24Data science as a pyramid
- 18:37The conditional expectation
- 22:28Applying Lusin's theorem
- 25:18Unifying methodology
- 29:08Practical modeling example
- 31:40Achieving explicable precision
- 35:07Mathematical convergence types
- 38:41Strategies for model selection
- 43:46Conclusion and implications
- 45:52Q&A session
related talks
37mEfficient Algorithms for Reliable Machine Learning
50mFrom Classical Statistics to Modern Machine Learning
59mTutorial: Statistical Learning Theory and Neural Networks I
01h 07mTheory of Computation II
01h 51mUnderstanding Neural Networks (Somewhat)
31mUncertainty quantification: How much can you trust your machine learning model?
36mQuantifying uncertainty in Machine Learning predictions
01h 12mRecent Progress in High-Dimensional Learning