Private Learning with Auxiliary Public Data
summary
A private predictor wraps any non‑private learner as a black-box transformation and uses subsample-and-aggregate / distance-to-instability plus the sparse vector technique to answer many online classification queries while spending privacy budget only on “unstable” queries. The construction splits the private training set into K parts, majority‑votes their classifiers, adds noise to the vote margin, and pays privacy only for ≈K^2/ε^2 unstable cases; empirical guarantees map the base learner’s error to a comparable private error and allow a knowledge‑transfer classifier whose sample complexity depends only on VC dimension.
AI-written summary · about
chapters (9)
- 0:00Introduction and goals
- 3:31Two-phase approach
- 8:34Design requirements
- 12:41Key construction results
- 16:52Distance to instability
- 22:14Transformation statement
- 29:40Agnostic PAC learning
- 36:15Algorithm technicalities
- 44:27Summary and discussion
related talks
01h 07mLarge-Scale Private Learning, Part II
17mData privacy and reliable learning
01h 02mLarge-Scale Private Learning, Part I
28mPrivacy preserving machine learning
48mSynthetic Data as an Enabler for Learning from Decentralized Private Data
01h 17mTutorial: Privacy, Part I
47mPrivate targeting strategies
36mPrivately Learning High-Dimensional Distributions