From Pandas to Polars Upgrading Your Data Workflow
summary
Compares Pandas and Polars by showing concrete edits on a 1 MB Strava GPX: Polars uses a Rust backend with Arrow, lazy evaluation and multithreading to avoid Python overhead, plus an optional GPU backend that gave ~10x speedup in tests. Demonstrates shrinking an int64 column to int16 to save memory, shows map_elements (Python callbacks) is costly while expression-based ops are 25x faster (965 μs vs 37 μs), and builds time-window aggregations that plot and work as expected.
AI-written summary · about
chapters (9)
- 0:00Introduction and speaker background
- 6:33Why switch from pandas to Polars
- 11:57Loading and inspecting Strava data
- 16:21Cleaning data and optimizing data types
- 29:02Handling missing data efficiently
- 33:20Chaining operations for readability
- 39:59Performance pitfalls and best practices
- 45:46Aggregations and visualizations
- 52:07Summary and key takeaways
related talks
01h 32mAn Introduction to Polars
28mExpressive & fast dataframes in Python with polars
38mExpressive and fast dataframes in Python with polars
32mPandas, Polars and the DataFrame Consortium.
25mIntegrating Rust with Python...
39mWhat we learned by converting a large codebase from Pandas to Polars
30mIs the great dataframe showdown finally over? Enter: polars
01h 16mPower up your Polars code with Polars extention