Data Mesh: Lessons from the trenches
summary
Large Fortune‑100 healthcare payer tries to move analytics to the cloud with a goal of ~1,000 assets migrated, but finds success only by building a custom self-serve platform on Azure that provisions Databricks, storage and isolated identity, exposes a CLI and a declarative spec, and enforces federated computational governance. Data is packaged as data product = data+code+policy+infra. The platform speeds new initiatives and turns governance from blocker to automated enabler, though enterprise-wide org change remains the hard part.
AI-written summary · about
chapters (16)
- 0:00Intro
- 2:40State of Data at Enterprise Pre Data Mesh
- 5:39How does Data Platform fit into the bigger
- 8:04Client's Measuring Success Heavy Focus on cloud migration
- 10:52Enabling analysts and data scientists Examples of questions facing the business
- 11:40Why Data Mesh? Approaches to extracting value from data
- 14:28Four Principles of Data Mesh
- 17:44Domain Oriented Decentralization Learnings and key factors
- 19:57Product discovery
- 22:28Policies Access Management, Encryption rules, De Identification patterns, etc.
- 23:12Code Standard patterns, flexible internals
- 23:58Infrastructure
- 28:23Data as a Product Learnings and key factors
- 33:06Data Product Experience Plane Series of product that manage the data product lifecycle
- 34:49Self Service Platform Learnings and key factors
- 39:08Automating governance under the platform
related talks
39mData Mesh: Why, What and How
20mEmbracing Data Mesh: A Modern Strategy for Scalable Data Management
31mData Mesh In Practice – How To Set Up A Data Driven Organization
48mData Mesh Paradigm Shift in Data Platform Architecture -
01h 00mData Mesh: Beyond the Data Lake
51mData Mesh – Successful with People, Process, and Technology
38mData Mesh, Now
38mLesson learned implementing a large-scale data mesh transformation