Keynote on Polars Plugins
summary
Polars is a Rust query engine that uses a persistent thread pool, work stealing and cache‑friendly Arrow buffers to do lazy, parallel execution and query optimizations. The uploader demonstrates running 80 million strings across many columns with three timings: a Python UDF at 22 s, an idiomatic Polars expression at 0.8 s, and a compiled plugin at 0.33 s. Polars exposes Polars Plugins so UDFs compile to native Rust performance and gain no GIL locking and optimizer benefits.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 2:00What is Polars
- 5:56Materialization
- 10:44Polars API
- 15:49API Promise
- 16:24Query Optimizer
- 17:24Query Optimizer Example
- 21:17Optimizations
- 22:49Parallelism
- 25:44Device the work
- 27:40Split the parallelism
- 28:13Performance
- 28:43TPCH Benchmark
- 29:19Plugins
- 31:42Code
- 33:10Clause Language
- 34:08Python
- 39:33Discord
- 40:46R Join
related talks
01h 16mPower up your Polars code with Polars extention
28mAdvanced Polars: Lazy Queries and Streaming Mode
01h 29mHow you (yes, you!) can write a Polars Plugin
38mExpressive and fast dataframes in Python with polars
28mExpressive & fast dataframes in Python with polars
25mIntegrating Rust with Python...
01h 32mAn Introduction to Polars
35mPolars GPU acceleration