Understanding and achieving a modern data architecture
summary
Describes a concrete AWS blueprint: an Amazon S3-based data lake with the Glue Data Catalog, purpose-built stores—Redshift (claims 3x price-performance), Athena ($5/TB scan), EMR (spot can save up to 80%)—and Glue for ETL plus serverless options. Introduces Lake Formation for unified governance with credential vending, tag-based access control, and row-level and cell-level permissions that enforce filters and audit via CloudTrail. Shows cross-account sharing patterns (hub‑and‑spoke, data mesh) and cites customers running exabyte-scale lakes.
AI-written summary · about
chapters (17)
- 0:00Introduction to modern data architecture
- 1:10Why modern data architectures matter
- 2:58Components of modern data architecture
- 4:46Data lakes on AWS
- 9:26Purpose-built data services
- 18:09Serverless analytics options
- 18:52Seamless data movement with Glue
- 23:11Unified data governance with Lake Formation
- 26:10Challenges in data lake permissions
- 30:52Lake Formation permissions model
- 34:27Row and cell-level permissions
- 37:27Tag-based access control
- 40:27Credential vending for S3 access
- 42:15Lake Formation Storage API
- 44:49Cross-account data sharing
- 49:10Common data sharing patterns
- 53:37Key takeaways and summary
related talks
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
28mTurn Data Into Insights
30mHarness the power of your data with the AWS Lake House Architecture
49mBuild modern data architecture and data mesh patterns on AWS
19mBuilding modern data lakes on Amazon S3
28mDeep Dive Into AWS Lake Formation
53mHow to build a modern data architecture
47mConstruyendo Lagos de Datos y Analíticos en AWS + Datos Abiertos