To Warehouse or to Lakehouse? And How Do Various Open Source Project Compare?
summary
Compares warehouses (Snowflake, Redshift, ClickHouse, Druid, Pinot) vs lakehouses built on Parquet/ORC with table formats like Hoodie, Iceberg, Delta Lake and query engines (Presto, Trino, Photon, Velox), arguing that the fastest decision comes from the underlying storage format and the typical read/write path. Claims there is a fundamental tradeoff: no system truly does OLTP and OLAP simultaneously, so pick warehouses for curated, low-latency analytic apps and lakehouses for cheaper, larger, ML-friendly datasets and flexible federation.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:35Agenda
- 4:09cheat code
- 5:31data modeling
- 6:34history lesson
- 11:06Cambrian explosion of databases
- 16:24Whats happened after that
- 18:35What is a database
- 25:30Schema Evolution
- 27:42Pros and Cons
- 33:03Conclusion
- 36:19Questions
related talks
24mData Mesh and Lakehouse
40mUnbundling of the Cloud Data Warehouse: Open Source Databases and Data Lakes
32mHow open source is re-shaping the cloud data warehouse landscape
25mBuild open format data lakehouse
58mUnderstanding Big Data for Software Engineers
34mData Lakehouse: Who, Why and How
21mWhy the Lakehouse?
34mCatalogs, Iceberg, and Delta: Building the Foundation for Lakehouse Analytics