Build modern data architecture and data mesh patterns on AWS
summary
Explains how to implement a data mesh on AWS by keeping data in place on S3 and sharing via AWS Lake Formation with tag-based, column- and cell-level permissions; shows concrete tools and flows: AWS Lake Formation tag-based access, Athena federated queries, Redshift Spectrum and Redshift ML, Glue/EMR/SageMaker for pipelines and models, QuickSight for dashboards. Describes cross-account resource shares, federated governance account, examples at Fannie Mae, J.P. Morgan and FINRA, and offers design/builder labs to produce working pipelines and ML models.
AI-written summary · about
chapters (9)
- 0:00Introduction and strategy
- 3:29Building on AWS
- 5:52Modern data patterns
- 13:28Deep dive into Lake Formation
- 17:45Architecture evolution
- 24:16Data mesh implementation
- 35:36Sharing and multi-account
- 40:55Federated query patterns
- 44:27Programs and resources
related talks
56mUnderstanding and achieving a modern data architecture
53mHow to build a modern data architecture
42mModern Data Strategy on AWS
28mTurn Data Into Insights
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
30mHarness the power of your data with the AWS Lake House Architecture
43mBoF: Data Mesh Community of Practice: What It Is, Why It's Hard, and How to Help...
28mDeep Dive Into AWS Lake Formation