Data Mesh – Successful with People, Process, and Technology
summary
Argues that data mesh is an organizational shift not just tech, moves analytical responsibility to the teams that own transactional systems, and requires each product team to publish a tested data product with metadata, API, code and infra as code. Recommends adding a data product owner and a data engineer per team, building a platform that provides access control, CI/CD, monitoring and billing, and running quarterly rotation of platform engineers into feature teams. Cites that 90% of data is recent while only 24% of firms are data driven and advocates starting with one use case and federated governance.
AI-written summary · about
chapters (17)
- 0:00Introduction and speaker background
- 3:26Data as oil versus wine metaphor
- 8:43Problems with traditional data lakes
- 11:55Core beliefs behind data mesh
- 13:18Data mesh principles explained
- 15:22Data producers and their role
- 17:10What is a data product?
- 18:36New roles for data teams
- 20:45Responsibilities of data producer teams
- 21:50Data consumers and their needs
- 24:16Role of the data mesh platform
- 26:46Balancing autonomy and alignment
- 28:15Federated governance in data mesh
- 31:05Where to start with data mesh
- 33:33Four steps to decentralize data lakes
- 34:58Practical advice and caution
- 35:27Q&A session begins
related talks
52mFrom Data Mess to Data Mesh: Navigating People, Process & Platforms
38mData Mesh, Now
31mData Mesh In Practice – How To Set Up A Data Driven Organization
23mHow To “Data Mesh”: What You Need To Know To Make It Real
20mEmbracing Data Mesh: A Modern Strategy for Scalable Data Management
43mUntangling Data Mesh: A paradigm shift in Big Data
43mWhen design meets data: Using Data Mesh for a consumer and business-driven data strategy
13mData Mesh is a Big Idea: Take Your Team From Vision To Reality