DuckDB Extension Development Workshop – Part 2
summary
Workshop shows how to implement a DuckDB table function called incremental_sequence in C++, covering bind, global init, per-thread local init and execute callbacks, plus a multi-threaded work queue that yields out 48-row chunks. The demo uses VS Code, debug builds and smart pointers, uses DuckDB’s sequence/flat vectors, adds a statistics callback to let the planner skip execution, and reports speedups (single-thread ~8s vs up to ~4.9x faster; claim of 100 billion rows in 3s presented).
AI-written summary · about
chapters (18)
- 0:00Introduction & Q&A
- 5:16Memory safety in C++ extensions
- 6:39Table functions and streaming
- 9:11Simple table function
- 11:38Bind and global init callbacks
- 17:42Global state and concurrency
- 22:19Executing table function
- 29:31Multi-threading in table function
- 36:14Progress and cardinality callbacks
- 40:08Column stats and query optimization
- 46:06Advanced table function features
- 47:32Publishing extensions
- 51:34Cross-platform development
- 54:06Designing useful extensions
- 57:17Extension ecosystem growth
- 58:22Table in-out functions & masking
- 1:01:22Future of extension APIs
- 1:03:51Closing remarks & Q&A
related talks
33mDuckDB – Overview and latest developments
40mDuckDB: Harnessing in-process analytics for data science and beyond
18mDuckDB Extensions
28mState of the Duck
19mIn-Process Analytical Data Management with DuckDB
34mData Wrangling with DuckDB
27mState of the Duck
06mDuckDB Applications in Information Retrieval