GeoClip: Geometry-Aware Clipping for Differentially Private SGD
summary
GeoClip transforms per-sample gradients into a learned basis (running noisy mean and covariance, eigendecomposition) and clips plus adds isotropic Gaussian noise there, then maps back, so the method claims lower added noise for the same clipping probability. The algorithm estimates the transform from previously released noisy gradients so it incurs no extra privacy cost, the optimal transform has a closed-form involving eigenvectors and a fourth-root eigenvalue scaling, and experiments (tabular, MNIST fine-tuning) show consistent gains—e.g., reaching low test error in about three epochs at ε≈0.17.
AI-written summary · about
chapters (10)
- 0:00GeoClip introduction
- 1:03Differential privacy in ML
- 3:31Standard clipping limits
- 6:24Introducing GeoClip
- 9:13Assumptions and theorem
- 13:11Comparison with Adam
- 17:50GeoClip implementation
- 19:57Synthetic data results
- 22:12MNIST and scalability
- 24:40Future directions
related talks
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics
01h 14mTutorial: Privacy, Part II
01h 07mLarge-Scale Private Learning, Part II
19mPractical privacy-preserving machine learning in Python
33mPrivacy of Decentralized Machine Learning
01h 02mLarge-Scale Private Learning, Part I
31mIs your model private?
36mPrivately Learning High-Dimensional Distributions