Postrevolutionary big data: Promoting the general welfare
summary
Comcast implements a “constitution” that keeps arbitrary data landing in the lake but requires publishers to register schemas on read with per-attribute documentation at onboarding, capture lineage and the identity of transformers, and expose a unified, searchable metadata+lineage repository. Duplication and putative lineage are detected with ML-based data deduplication that finds semantic equivalence between tables. The result aims to restore discoverability and cross-silo integration while preserving self‑service.
AI-written summary · about
related talks
37mThe perils of building a democratic data platform
01h 00mData Mesh: Beyond the Data Lake
27mLive in the Data Wild West: The Data Contracts Sheriff
37mData Swamp to Data Swimming Pool: Cleaning up the Mess
24mData’s Rotten Scoundrels – How To Avoid Becoming One
18mLarge-Scale Data Graph: Scale and Optimize Privacy & Security in Offline Data Systems
50mData Privacy Concerns - A Risk Based Approach to De-Identification
59mThe Great Data Debate