Harness the power of your data with the AWS Lake House Architecture

on YouTube ↗30m

" summary

AWS Lake Formation gives S3 data lakes govern tables with metadata manifests so updates move only metadata not petabytes, supports ACID transactions (concurrent inserts/updates/deletes), row-level security via SQL predicates, and time travel for previous versions. It integrates with Glue, Athena, QuickSight and SageMaker, keeps data in open formats like Parquet, and customers run tens of thousands of lakes holding petabytes. New acceleration and storage-optimizer features compact files and push down predicates to speed queries and reduce costs.

AI-written summary · about

" chapters (24)

1. 0:00Intro
2. 0:50In the past, decision making... ... revolved around the enterprise data warehouse
3. 1:36Data no longer fits
4. 2:31Broader workloads
5. 3:48Data lake: The new information hub
6. 4:10Customers moving to Lake House architecture
7. 5:17Lake House architecture on AWS
8. 6:53AWS Lake Formation solution stack
9. 9:33Ingesting data with blueprints
10. 9:55Securing sensitive data with column permissions
11. 10:12Recent AWS Lake Formation innovations
12. 11:23New AWS Lake Formation update and access APIs Open and public Build your own application
13. 12:15Why transactions? Continuous ingest and updates Privacy regulations (GDPR, CCPA)
14. 13:35Introducing Governed Tables
15. 15:53Governed Tables: Versions and deltas
16. 17:11Example - Transactions and isolation
17. 19:20Transactions make data lakes trustworthy
18. 20:15AWS Lake Formation row-level security
19. 21:14Example of row-level security: Setup
20. 21:52Example of granting row-level permissions Specify the principal
21. 22:17Example: Compute admin queries the data lake
22. 22:30Example: Storage admin queries the data lake
23. 23:27Common data sharing topologies
24. 24:41AWS Lake Formation started as...

" related talks

• 28mDeep Dive Into AWS Lake FormationSyed Jaffry · AWS Summit Online 2020
• 28mTurn Data Into InsightsRoy Hasson · AWS AMER Summit May 2021
• 56mUnderstanding and achieving a modern data architectureChanakya Damarla · AWS Summit San Francisco 2022
• 43mManaging a Data Lake on AWSAlexander Günsche · DevOps Barcelona 2024
• 34mBest Practices for Implementing a Data Lake in Amazon S3Kumar Nachiketa · AWS Summit Online 2020
• 59mModern data lakes with Lake Formation, Amazon Redshift & AWS GlueRaghavarao Sodabathaina, Changbin Gong · AWS Summit Atlanta 2022
• 47mConstruyendo Lagos de Datos y Analíticos en AWS + Datos AbiertosAngelo Carvalho · AWS Public Sector Summit Mexico City 2020
• 49mBuild modern data architecture and data mesh patterns on AWSFabrizio Napolitano, Prasanna Ramaswamy · AWS Summit Ottawa 2022