Building an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
summary
Shows how to build an open lakehouse end-to-end: create an Apache Iceberg table from Apache Spark, insert and merge rows, inspect table snapshots for time travel, then expose and join that same table live through Dremio (no data copied). Demo runs inserts, a merge and a UI join, and queries from Tableau via direct query. Covers Iceberg vs Delta contributors, Arctic for multi‑statement transactions and housekeeping, and a free Community Edition to try.
AI-written summary · about
chapters (10)
- 0:00Start
- 1:26What is an "Open Data Lakehouse"?
- 2:49The evolution to the Data Lakehouse
- 9:24Table format adoption
- 11:20How Is Dremio Different From Traditional SQL Engines?
- 13:48Typical components of an Open Data Lakehouse
- 16:56Using Dremio with Iceberg - CREATE TABLE
- 18:03Using Dremio with Iceberg - MERGE new data
- 19:55Demo
- 33:53From Data Warehouse to Data Lakehouse
related talks
45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
13mBuild your open data lakehouse with Dremio and AWS (PRT085)
25mBuild open format data lakehouse
01h 12mHands-on with Apache Iceberg
51mHow to Build a Modern Data Lakehouse?
28mStreaming Lakehouse with Kafka, Flink and Iceberg
50mDesigning a scalable data platform based on Apache Spark/Iceberg
42mBuilding a Lakehouse data platform using Delta Lake, PySpark, and Trino