We rewrote tsfresh in Polars and why you should too
summary
A full rewrite of tsfresh becomes functime built on Polars and Rust and it actually ships hundreds of reimplemented features that run much faster and use less memory the authors show up to 50x speedups on single feature vectors and microbenchmarks with 9M rows and dramatic group by gains (they report around 200x in some grouped S&P500 tests) parallelizing on Rayon so no Spark or Dask cluster is needed and unit tests plus pluginable Rust expressions validate correctness.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:48What is tsfresh and limits
- 3:35Panel data format
- 5:02Why tsfresh needed rewrite
- 7:06Funtime and Polars intro
- 9:32Motivation for rewrite
- 10:34Code example
- 11:59Benchmarking tsfresh vs Funtime
- 13:48Top 7 learnings
- 16:36Rust and Polars for science
- 18:26Improving tsfresh API
- 20:32Polars plugin system
- 21:57Development approach
- 22:38Why use Funtime
- 24:25Q&A
related talks
38mExpressive and fast dataframes in Python with polars
28mExpressive & fast dataframes in Python with polars
25mIntegrating Rust with Python...
01h 16mPower up your Polars code with Polars extention
02h 14mGetting Started with Polars
30mIs the great dataframe showdown finally over? Enter: polars
01h 32mAn Introduction to Polars
49mKeynote on Polars Plugins