The Emperor's New Embeddings: Obfuscating ML Inputs Doesn't Provide Privacy
summary
Claims that token or embedding obfuscation gives inference privacy fail: applying the data processing inequality shows transformed inputs cannot hide more information than they contain, large language models routinely reproduce prompt content, and reported reductions rely on loose entropy bounds or intractable adversary models that make bit estimates meaningless. Demonstrations of partial utility often arise from task narrowing, not true protection. Practical safeguards are confidential computing and cryptographic approaches like homomorphic encryption or secure MPC combined with differential privacy.
AI-written summary · about
chapters (13)
- 0:00Input obfuscation in ML
- 1:14What is input obfuscation
- 3:23Input obfuscation claims
- 5:07Data processing inequality
- 7:35LLMs preserve input info
- 8:57Adversaries process obfuscated inputs
- 11:04Information in a name
- 13:10Obfuscation paper flaws
- 14:58Flaws in obfuscation techniques
- 17:27Translation tests fail
- 18:32Data minimization use cases
- 19:37Differential privacy tools
- 20:59Q&A
related talks
01h 28mPPML: Machine Learning on Data you cannot see
38mPersonal data de-identification for data science tasks
28mPrivacy preserving machine learning
42mPrivacy Challenges in Machine Learning
26mDoes using data mean giving up privacy?
31mScaling privacy embedded distributed ML systems
22mPrivacy-preserving Computation in Plain Sight
29mPrivacy-preserving methods: Building secure projects