Run Numba functions in SQLite: WTF?
summary
An experimental library called slumba embeds Numba-generated C-callable functions into SQLite UDFs, aggregates and window functions by emitting LLVM/JIT code and wiring it to the SQLite C API. Claims: whole-table aggregations run ~5x faster, low-cardinality GROUP BYs ~2x, high-cardinality ~1.5x, scalar UDFs show little gain. Limitations: no heap/object work, null/type fiddling, risky unsafe casts. Demo registers aggregates and runs correctly; project remains hacky and experimental.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:07Extending SQLite with Python UDFs
- 6:07User-defined aggregates in C
- 10:30What is Numba
- 13:21Goal: Numba JIT in SQLite
- 17:21Type inference for Numba in SQLite
- 20:39Performance benchmarks
- 23:15Demo: Numba aggregates in SQLite
- 24:21Q&A
related talks
32mWebAssembly functions for your SQLite-compatible database
47mLumoSQL - updating SQLite for the modern age
30mUse SQLite in production
30mML meets NoSQL: Integrating Python user-defined functions (UDFs) with N1QL for analytics
27mUsing embedded SQL engines for plotting massive datasets on a laptop
14mUDF to SQL Translation through Compositional Lazy Inductive Synthesis
31mExtending SQL Databases with Python
24mDeveloping DB Extensions in Python: Examples for SQLite, DuckDB & PostgreSQL