Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
summary
Proves a uniform‑convergence bound for interpolators in Gaussian linear regression: test loss ≤ (Gaussian‑width(K))^2 / n up to small additive/multiplicative terms, with a covariance split that recovers Bartlett et al. benign‑overfitting conditions. Analysis uses Gaussian width (equivalently Rademacher complexity here), a Convex Gaussian Min‑Max reduction (Gordon’s theorem), and a Lagrange‑multiplier norm bound to control the minimum‑norm and even L1 interpolators. Also gives an optimistic‑rate (square‑root) variant that bridges nonzero training loss.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:52Introductions
- 6:53Double Descent Curve
- 8:50Matrix Notation
- 25:27Generalization Bound
- 27:02Random Macro Complexity
- 42:38Rate of Convergence
- 48:39Upper Bound on the Norm of the Minimum Normal Interpolator
- 48:59The Generalization Bound
related talks
01h 18mReconsidering Overfitting in the Age of Overparameterized Models
48mBenign Overfitting in Linear Prediction
01h 22mGeneralization IV
55mThe Devil is in the Tails and Other Stories of Interpolation
29mImplicit Regularization in Nonconvex Statistical Estimation
32mQuantitative Correlation Inequalities via Semigroup Interpolation
01h 23mImplicit Regularization II
01h 02mPrediction, Generalization, Complexity: Revisiting the Classical View from Statistics Part 2