Fireside Chat
summary
Explains why boosting drives weights toward infinity yet still generalizes: PAC-style proofs predict overfitting but experiments do not, and margin theory offers a concrete explanation. Covers two analytic approaches—fixed training-set bounds and a streaming IID proof—plus connections to neural nets via weight norms and to dataset compression properties. Gives explicit examples (astral space limits and a train-ride theorem), recounts which theorems were proved and which empirical surprises forced new theory, and mentions tools used rather than vague promises.
AI-written summary · about
chapters (8)
- 0:00Origins of margin theory
- 3:33Theoretical expectations
- 4:52Theory versus practice
- 7:13Research motivation and habits
- 11:42Advice for students
- 12:46Evolution of machine learning
- 17:03Navigating academic pressures
- 19:01Career reflections
related talks
01h 10mModern paradigms of generalization, the heliocentric model of Aristarchus,...
56mLightning Talks!
01h 21mTutorial: Implicit Bias I
09mLightning Talk: Intellectual Cognitive Development in Artificial Intelligence
01h 02mPrediction, Generalization, Complexity: Revisiting the Classical View from Statistics Part 2
01h 04m"More is Different." Also in Computing?
36mScaling Laws, Emergent Behaviors, and AI Democratization
01h 18mPrediction, Generalization, Complexity: Revisiting the Classical View from Statistics Part 1