Supercharging your pandas workflows with Modin
summary
Claim: Modin is a drop-in replacement for pandas that you pip install and run with the same syntax. Benchmarks show concrete gains: 3x faster S3 reads, 2x faster preprocessing, 8x speedups in a bank risk pipeline, 5x at Spotify, and a hugging face sentiment pipeline moving from a crashing 5‑hour run to ~20 minutes (~12x). Implementation detail: it spills to disk to avoid OOM and supports >90% pandas API while mapping 600+ operators for distributed execution.
AI-written summary · about
chapters (6)
- 0:00Modin introduction
- 3:15Pandas scaling struggles
- 8:18Modin performance gains
- 10:52Demo: Hugging Face speedup
- 19:06Modin architecture future
- 21:34Q&A
related talks
38mScaling Interactive Pandas Workflows with Modin
29mLearning to Scale Data Science, Machine Learning, and Pandas with Ray and Modin
26mBodo: Supercomputing-Like Performance and Scale for Python/Pandas
05mPandas on DuckDB with Ponder
39mAccelerated Data Science Libraries
32mPandas May Be Slow, but Pandas Doesn't Have to Be
29mMaking Pandas Fly
28mIdiomatic Pandas