Keeping data private in real-time pipelines
summary
Covers concrete patterns for streaming privacy using Apache Kafka, Apache Flink, and Apache Iceberg: masking is trivial but irreversible, tokenization requires a private token vault and deterministic vs random tokens trade utility for risk, hashing is easily reversible (New York taxi example), k-anonymity and bucketing reduce fingerprinting but are hard to tune, and adding noise or poisoning helps ML training; case studies include Netflix’s ~100 million ratings leak and AOL’s 20M search logs, plus homomorphic encryption as an expensive future option.
AI-written summary · about
chapters (7)
- 0:00Introduction and data privacy risks
- 6:48Foundation for data privacy control
- 13:15Data masking and tokenization
- 20:18Re-identification risks in anonymized data
- 27:49Advanced privacy techniques
- 33:50Balancing privacy and data utility
- 34:31Q&A on tokenization and privacy
related talks
33mKeeping data private in real-time pipelines
45mDeep Dive into Data Streaming Security
26mDefining a Data Masking Framework at Scale
50mPrivacy by design
27mEnd-to-End Encryption for Streaming Data Pipelines
39mPrivacy-preserving Data Sharing
56mDeep dive into data streaming security
32mThe Dangerous Data Anonymization - Keynote