Practical Difficulties in the Application of Differential Privacy for the Release of Large-Scale...
summary
Census builds a production disclosure avoidance system (DAS) that uses the high‑dimensional matrix mechanism and top‑down optimization to release microdata (MDF) for ~300M person and ~100M household records across 10 million blocks, targeting PL94, SF1, SF2, AINSF tables. Implementation is Python/NumPy/Spark; it solves huge simultaneous inference problems but faces non‑end‑to‑end DP due to invariants, struggles to estimate end‑to‑end error for billions of tabulations, and encounters practical failures like a Mersenne Twister vulnerability and Spark‑partitioned correlated noise.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:12What Is the 2020 Disclosure Avoidance System
- 2:59Census Edited File
- 3:25Invariants
- 22:53The Data Dictionary
- 28:29The High Dimensional Matrix Mechanism
- 31:05Error Estimation
- 31:55Why Is It Hard To Do this in Practice
- 38:17Policy Issues
related talks
20mShare data without revealing personal information
15mDifferentially Private Algorithms for 2020 Decennial Census Detailed DHC Race & Ethnicity
19mPractical Considerations for Differential Privacy
47mDeploying DP: Experiences from the Front Lines and Open Challenges
45mPractical Experience with Making Synthetic Data Differentially Private
55mPANEL: Where Are Statistical Agencies Going?
27mYou've got trust issues, we've got solutions: Differential Privacy
01h 00mStepping-up: The Census Bureau Sets an Example of How to Be a Good Data Steward in the 21st Century