Building an End-to-End De-Identification Pipeline for Advertising Activity Data at...
summary
LinkedIn builds an end-to-end deidentification pipeline that processes billions of events a day using both streaming (Samza) and batch paths with QoS tiers to avoid freshness gaps. Schema-driven field annotations force privacy actions or automatic failures. Numeric fields use clamping (dwell time 300 ms–300 s), Gaussian local DP with epsilon=4 and metric DP for timestamps; identifiers become stable tokens via a threshold oblivious PRF implemented as client randomized encodings, key servers and gRPC services. Outcome: maintained SLAs, reduced PII retention, and onboarded reporting, billing and reach with continuous risk monitoring.
AI-written summary · about
chapters (13)
- 0:00Trust and data privacy at LinkedIn
- 1:11De-identification critical for ads
- 2:38Ads data flow through systems
- 3:39De-identification pipeline requirements
- 4:22Streaming vs batch architectures
- 5:27Stream processing challenges
- 6:53Quality-of-service pipeline
- 7:59Schema-driven privacy enforcement
- 9:02Privatization mechanisms
- 10:05Differential privacy for timestamps
- 12:11Member ID tokens
- 13:57De-identification system results
- 14:39Future work and improvements
related talks
50mData Privacy Concerns - A Risk Based Approach to De-Identification
09mPrivacy Analytics and De-Identification
29mPersonal Data Pseudonymization Versus Anonymization In The Age Of AI & Big Data
29mOperationalizing Data In Support Of Privacy Regulations
33mKeeping data private in real-time pipelines
50mPrivacy by design
40mData Privacy and Security by Design
40mProtecting sensitive data in huge datasets #bbuzz