Why DuckDB could be the engine for your Data Lake?
summary
Introduces DuckDB as an alternative engine for datalakes/lakehouses, arguing it enables SQL processing of large datasets stored on object storage using limited resources while preserving data sovereignty and GDPR control. Shows concrete examples, code snippets and benchmarks to demonstrate how to build and operate such a setup, aimed at both beginners and experienced data engineers.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:09What is a Data Lake?
- 7:17Data Lake implementations
- 10:44Transforming data simply
- 14:11DuckDB overview
- 20:10Demo with DuckDB
- 39:55Going further with DuckDB
- 41:02Q&A
related talks
40mDuckDB Meets Data Lakes
47mPlay Data Engineer Locally with DuckDB
38mFrom Big Data to DuckDB: Ryan Boyd (MotherDuck) on Hyper-Tenancy, Duck Lake, and Making Data Fun.
55mDuckLake: Learning from Cloud Data Warehouses to Build a Robust “Lakehouse”
26mBuilding Data Lake Using DuckDB
28mAnalyze data easily with duckdb - and the implications on data architectures
27mThe Duck and the DataFrame: A Data Engineer’s Journey with DuckDB
51mMotherduck: Data Analytics in the Post Big-Data Era