Increasing Participant Agency by Understanding the Impact of Training Data
summary
Shows a scalable method to compute the meta-gradient—a gradient of final model behavior with respect to continuous training-data parameters—by checkpointing/replaying training (log T memory, ~T log T extra work). Uses that predictive model of data to drive machine unlearning by distilling predicted retrained-model outputs back into parameters (much faster and closer to the oracle than naïve gradient-ascent unlearning). Uses iterative meta-gradient search for data editing / poisoning: poisoning 1% of CIFAR-10 drops accuracy ~14% (vs ~1% for prior attacks).
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:05Motivating predictive models
- 3:11Meta-gradient computation
- 19:05Validating predictive model
- 27:01Machine unlearning
- 38:22Q&A
- 39:05Data editing for agency
- 45:04Summary
related talks
01h 04mPredicting and optimizing the behavior of large ML models
50mFlexible Neural Networks and the Frontiers of Meta-Learning
47mMeta-learning of Optimizers and Update Rules
23mTTT, TAO, δ-mem: When to Spend a Gradient
39mDuplicating Black Box Machine Learning Models
41mOptimization Algorithms for Deep Learning
21mOperationalizing ML Training Infra at Meta Scale
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics