Cocoon: A System Architecture for Differentially Private Training with Correlated Noises
Talkat 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26) · 2026-07-13..07-15·
summary
Cocoon presents a system that stores and computes correlated noises across GPU, CPU, and a CXL near‑memory processor to avoid the 14× slowdowns observed when a noise history spills from GPU memory. It targets models with huge embedding tables (≈99% of params; example: 33M entries with <1M touched per iteration), uses near-memory processing on an FPGA NMP prototype with 256 GB CXL, and applies premputation and recalling for sparse embeddings, yielding 1.23–10.82× speedups in real experiments.
AI-written summary · about
chapters (12)
- 0:00Cocoon intro and privacy in ML
- 1:03Why privacy matters in ML
- 2:09DP-SGD and correlated noise
- 3:56System overheads of correlated noise
- 5:24Memory pressure in economical setups
- 7:07Overhead severity in recommendation models
- 8:57Dense models and memory bottlenecks
- 9:39Cocoon’s two key solutions
- 11:25Near-memory processing and history distribution
- 12:08Embedding optimizations with precomputation
- 13:54Performance evaluation
- 14:34Cocoon’s speedup over baselines
related talks
22mCXL-ANNS: Software-Hardware Collaborative Memory Disaggregation and Computation...
15mA General Framework for Communication-Efficient Distributed...
28mFrom Pixels To Agents: Optimizing On-Device Performance of Confidential Computing in AI Evolution
17mApproximate Caching for Efficiently Serving Text-to-Image Diffusion Models
16mZico: Efficient GPU Memory Sharing for Concurrent DNN Training
16mAccelerating Design Space Exploration for LLM Training Systems with Multi-experiment...
17mZEN: Empowering Distributed Training with Sparsity-driven Data Synchronization
23mBatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning