Solving the challenges of adding transactional support to Data Lakes
summary
Overview of data-management frameworks that add transactional features to data lakes on AWS, comparing Apache Hudi, Apache Iceberg and AWS Lake Formation governed tables and how they integrate with the AWS analytics ecosystem. Covers problems like change-data-capture, GDPR-style deletions, ACID-style UPSERT/DELETE, time-travel queries, unifying streaming and historical analytics, concurrency isolation and performance improvements.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:08Transactional challenges in data lakes
- 11:05Case study example
- 12:29Frameworks for solutions
- 22:04Apache Hudi and Apache Iceberg
- 36:28AWS ecosystem integration
- 43:13Framework selection criteria
- 47:06Table migration and coexistence
- 49:30Summary and resources
related talks
41mTransactional data lakes for your analytical workloads
01h 03mReinvente su estrategia de datos con Lagos de Datos Transaccionales
22mThe Who, What, and Why of Data Lake Table Formats
30mHarness the power of your data with the AWS Lake House Architecture
31mData Warehouses Meet Data Lakes
34mData Lakehouse: Who, Why and How
43mManaging a Data Lake on AWS
25mBuild open format data lakehouse