Polars 1.0 and beyond
summary
Polars reaches the 1.0 milestone (July 1, 2024) and adds GPU acceleration (August) via collaboration with NVIDIA Rapids, keeping identical semantics and a hybrid engine with seamless CPU fallback. Benchmarks show up to 13x speedup on TPCH-derived queries where CPU work dominates. Engineering work ships a SIMD-accelerated Parquet/IPC reader, a new non-equi join algorithm that is 100x faster than the previous approach, IO plugins, plotting backend, and an in-progress cloud/streaming engine.
AI-written summary · about
chapters (14)
- 0:00Introduction to Polars 1.0
- 1:08What Polars is and query engine
- 5:03GPU acceleration in Polars
- 13:18Polars 1.0 release and API
- 15:50Recent performance improvements
- 21:12Polars plugin system and IO
- 23:44Efficient data loading and caching
- 26:17Correctness and error handling
- 27:41Polars streaming engine
- 29:32Q&A on dependencies and SQL
- 30:34Q&A on supply chain security
- 31:39Q&A on SQL compatibility
- 33:04Q&A on visualization backend
- 35:11Q&A on async runtime
related talks
44mPolars; DataFrames in the multi-core era
01h 32mAn Introduction to Polars
31mPolars on Spark
39mDesigning for high-performance execution with Arrow and Polars
01h 22mAn Introduction to Pandas 2, Polars, and DuckDB
02h 14mGetting Started with Polars
36mHow Narwhals brings Polars, DuckDB, PyArrow, & pandas together
34mBridging Pandas and Polars