Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS
summary
Claim: the reproducing-kernel Hilbert space of the rotation-invariant neural tangent kernel on the unit sphere equals that of the Laplace kernel (e^{-c||x-y||}) as a set, and this equality is depth-independent so extra layers do not enlarge RKHS. Proof analyzes decay of power series coefficients via complex analysis locating nearest singularities. A second result shows smaller smoothness parameter γ in the exponential-power kernel yields a larger RKHS, proved using an inverse Laplace transform argument.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:30Kernel Review
- 1:49Neural Tangent Kernel
- 2:40RKHS
- 4:04Smoothness Size
- 5:37Feature Engineering
- 8:08Classification
- 9:18Model
- 10:31Results
related talks
01h 14mRecent Developments in Over-parametrized Neural Networks, Part II
01h 00mA Theoretical Framework of Convolutional Kernels on Image Datasets
33mThe Power and Limitations of Kernel Learning
01h 22mGeneralization IV
42mToward Understanding the Extrapolation of Nonlinear Models to Unseen Domains or the Whole Domain
38mOn Laplacian Eigenmaps for Dimensionality Reduction
53mConnection Laplacian, Hodge Laplacian, and Tensor Laplacian of a Graph
50mLearning and Generalization in Over-parametrized Neural Networks, Going Beyond Kernels