Privacy Amplification from Structured Algorithmic Randomness
summary
Claim: inherent algorithmic randomness in compression, model partitioning and structured subsampling measurably amplifies privacy so external Gaussian noise can be reduced. Tools: analysis of a Rényi divergence bound between Gaussian mixtures and a pure Gaussian plus order-wise MSE bounds showing the error is max(of compression term, privacy term). Results: 1M-parameter runs compressing gradients by 1,600× keep accuracy equal to no-compression; 40M model splitting into 3 submodels recovers roughly half the accuracy lost to DP noise. random gradient subsampling balanced iteration subsampling
AI-written summary · about
chapters (11)
- 0:00Federated learning basics
- 2:01Privacy amplification via randomness
- 5:06Randomness in gradient compression
- 10:23Federated learning with compression
- 23:41Combining compression and privacy without accuracy loss
- 29:27Model splitting for privacy
- 32:15Model splitting analysis
- 37:18Experimental validation
- 44:15Open problems in dropout
- 45:25Structured sampling for privacy
- 47:47Summary and future directions
related talks
50mPrivacy Amplification by Sampling and Renyi Differential Privacy
35mPrivacy amplification by random allocation
01h 14mTutorial: Privacy, Part II
01h 02mLarge-Scale Private Learning, Part I
43mPrivacy Amplification by Sampling in Practice
01h 07mLarge-Scale Private Learning, Part II
01h 14mTutorial: Privacy, Part III
26mAaron Roth, Associate Professor, University of Pennsylvania, NYC 2017