DuckDB: Crunching Data Anywhere, From Laptops to Servers
summary
DuckDB claims to load 15 GB of CSV into its columnar format in 11 seconds (15 GB in 11s) and run an 11–15M‑row aggregation in 0.2s, scale to ~600 GB CSV with a 16‑minute load, and beat a 2‑node Spark run on a 40 GB TPC‑H Parquet set (1:16 vs ~8 minutes). It is in‑process, uses columnar storage, vectorized execution and zone maps (vectorized execution), supports Parquet/CSV/JSON/Delta/Iceberg/httpfs/S3 and WebAssembly, and offers zero‑copy pandas integration.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:23What is DuckDB?
- 2:48Demo
- 5:09What is DuckDB? continued
- 6:48Demo
- 11:09In-process architecture
- 13:44Internals: Storage & execution
- 19:21Portability
- 20:56Ergonomic
- 24:48State of DuckDB
- 25:58Extensions
- 27:10Use cases
- 31:34Limitations
- 32:57Business model
- 35:03Summary
- 36:06Outro
related talks
40mDuckDB: Harnessing in-process analytics for data science and beyond
31mData wrangling like a boss with Duckdb
58mData Wrangling [for Python or R] Like a Boss With DuckDB
01h 02mAnalytics for not-so-big data with DuckDB
34mData Wrangling with DuckDB
47mGábor Szárnyas | Data Track | Compass AI & Data Summit 2025
28mAnalyze data easily with duckdb - and the implications on data architectures
25mProcessing Biggish Data with DuckDB and Python