Data preparation and analytics on your laptop with DuckDB
summary
Claims are concrete: DuckDB reads a 160 GB compressed eBird file with read_csv and casts messy columns into types, loads into MotherDuck with an initial ingest that takes about an hour and produces a table of 1.7 billion rows. Single queries saturate a node and return results in milliseconds — a demo query hits ~14 ms single-query latency. The demo builds temp tables, computes H3 hex buckets, plots migrations with Plotly, and generates SQL from natural language via natural-language-to-SQL via MotherDuck — and it works end-to-end.
AI-written summary · about
chapters (12)
- 0:00DuckDB and big data trends
- 3:00Single-node analytics with DuckDB
- 5:30Large datasets on a laptop
- 7:57Connecting to DuckDB
- 12:15eBird dataset structure
- 14:27Most observed birds in WA
- 16:56Snow goose migration patterns
- 20:48Long-distance migrating birds
- 22:35Natural language queries
- 25:25LLMs with database queries
- 26:29Summary
- 26:55Q&A
related talks
01h 02mAnalytics for not-so-big data with DuckDB
34mData Wrangling with DuckDB
27mThe Duck and the DataFrame: A Data Engineer’s Journey with DuckDB
45mAnalytics for not so big data with DuckDB
27mIn Process Analytical Data Management with DuckDB
40mDuckDB: Harnessing in-process analytics for data science and beyond
37mDuckDB: Crunching Data Anywhere, From Laptops to Servers
27mMotherDuck: Taking flight with interactive analytics