Stargazing into AI Usage with Differential Privacy
summary
Urania implements end-to-end differential privacy for AI usage logs by rewriting chats with an LLM, embedding the summaries, running DP k‑means clustering, and using partition selection (Laplace noise + thresholding) to privately surface frequent keywords that an LLM expands into cluster descriptions; it runs on large corpora (WildChat demo finds Python code‑generation clusters) but hits a clear utility cliff as epsilon shrinks. A follow‑up idea, Calliope, replaces geometric embeddings with LLM‑native hierarchical clustering (subsample + classify / Hot Potato), trading scalability and cost against higher text‑domain utility.
AI-written summary · about
chapters (12)
- 0:00AI usage with differential privacy
- 1:45Problem statement and motivations
- 4:14Threat models and trust boundaries
- 6:44Clio method overview
- 12:26Urania with differential privacy
- 14:57Live demo and architecture
- 17:28Embedding model critiques
- 18:55LLM-only approach limits
- 21:46Keyword clustering with privacy
- 27:05Summary clustering techniques
- 28:34Measuring clustering utility
- 30:47Open problems in AI usage
related talks
13mA DP Framework for Gaining Insights into AI Chatbot Use
10mThrough the Lens of LLMs: Unveiling Differential Privacy Challenges
31mScaling privacy embedded distributed ML systems
01h 20mUnlocking societal benefits of AI with privacy protective technologies.
58mAI-Enhanced Big Data: Integrating Private LLMs and Vector Databases
45mCrypto-AI: A Guide to Building Private and Decentralized AI Systems
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
14mEstablishing Privacy Metrics for Genomic Data Analysis