Data Wrangling with DuckDB
summary
DuckDB is described as an MIT-licensed embedded analytical SQL database that runs in-process, uses Arrow to shuttle data to and from Pandas and Polars, and claims big wins on aggregations — a demo shows ~2–3 seconds vs 50 seconds for a 100 million‑row groupby where Pandas slows dramatically. Demos include CSV import robustness, Parquet roundtrips, SQLite extension reads, a web/CLI/UI, an HTTPS extension for one‑line pipelines, benchmark plots (one anomaly noted), and working interactive examples.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:01What is DuckDB
- 6:12Why DuckDB matters
- 9:23DuckDB vs alternatives
- 10:48Using DuckDB in practice
- 12:59Performance comparison with Pandas
- 15:10Big data landscape
- 19:00Live demo: import and analyze
- 23:10SQLite and extensions
- 24:57Benchmark results
- 27:48One-line pipelines
- 29:15Q&A
related talks
40mDuckDB: Harnessing in-process analytics for data science and beyond
27mIn Process Analytical Data Management with DuckDB
28mAnalyze data easily with duckdb - and the implications on data architectures
01h 02mAnalytics for not-so-big data with DuckDB
30mDuckDB: Bringing analytical SQL directly to your Python shell
19mIn-Process Analytical Data Management with DuckDB
58mData Wrangling [for Python or R] Like a Boss With DuckDB
39mPython and SQL: Better Together, Powered by DuckDB