Unforgettable, that's what you are: Evaluating Machine Unlearning and Forgetting
summary
Shows that large models memorize both common and novel (outlier) examples and that attackers can use shadow models + discriminators for membership inference and even do image inpainting to exfiltrate training data. Model-distance checks fail due to permutation invariance, while practical unlearning options include CISA sharding with mixture-of-experts, gradient-ascent forgetting with curated retain sets or pseudolabeling, and token-probability LRA tests (Microsoft’s Harry Potter experiments over 20–120 steps). Warns unlearning leaves artifacts, raises compute costs (imagine 100 deletes/week or frequent retrains) and amplifies risk for outlier populations via the privacy onion effect.
AI-written summary · about
chapters (14)
- 0:00Machine unlearning intro
- 1:11Memorization in deep learning
- 6:14Privacy risks of memorized data
- 9:46Inpainting attacks for data exfiltration
- 13:16Machine unlearning methods
- 15:49Measuring unlearning success
- 18:17Task-specific unlearning
- 20:27Influence and retention in unlearning
- 22:10Training-based unlearning
- 23:38Gradient-based unlearning
- 25:46Scalability in unlearning
- 26:50Unlearning limitations and risks
- 28:57Rethinking learning and privacy
- 29:39Privacy as a human right
related talks
15mLearning and Unlearning Your Data in Federated Settings
32mThe Importance of Forgetting for Both Humans & Machines
18mRectifying Privacy and Efficacy Measurements in Machine Unlearning...
31mGetting AI to Forget About You: Data Deletion From Machine Learning Models
01h 04mCan you trust your (large language) model?
26mExact Unlearning of Finetuning Data via Model Merging at Scale
29mBalancing Thin Line Between Data Intelligence & Privacy
54mKeynote: Can you trust your (large language) model?