DuckDB beyond the notebook
summary
DuckDB is shown running in the browser via DuckDB browser WebAssembly, loading Parquet/CSV from S3 or local files and connecting to Arrow/Polars without copies. A Lambda example performs simple ETL and writes to S3 in about lambda ETL in ~10s. Replacing open‑source PySpark with SQLFrame plus DuckDB converts a 1M‑row job (Spark cold ~7s, warm ~4s) to much faster execution — 1M-row job via SQLFrame beats open-source PySpark. Metadata moves into Duck Lake metadata DB + quark remote protocol, enabling remote attach and lightweight serverless architectures.
AI-written summary · about
chapters (14)
- 0:00Introduction to DuckDB
- 1:12DuckDB basics and CLI demo
- 3:04Querying remote and cloud data
- 5:33Log file analytics with DuckDB
- 7:22DuckDB for AI data access
- 8:46DuckDB in AWS Lambda for ETL
- 12:01Replacing Spark with SQLFrame
- 15:27Building analytical apps with DuckDB
- 19:24Beyond three-tier architecture
- 24:47DuckDB in the browser
- 25:11DuckLake and metadata databases
- 26:57Quack protocol for remote connections
- 34:03Use cases and future of DuckDB
- 38:13Key takeaways and conclusion
related talks
40mDuckDB: Harnessing in-process analytics for data science and beyond
28mAnalyze data easily with duckdb - and the implications on data architectures
27mIn Process Analytical Data Management with DuckDB
34mData Wrangling with DuckDB
01h 02mAnalytics for not-so-big data with DuckDB
27mThe Duck and the DataFrame: A Data Engineer’s Journey with DuckDB
05mDuckDB Powering Interactive Notebooks
19mIn-Process Analytical Data Management with DuckDB