Fast Pandas Performance Tricks You Wish You Knew Earlier
Talkat PythonAsia 2026 · on Saturday, 2026-03-21· at 15:30· Yuchengco Hall 5th Flr. Y507 (Workshop Room 1)·
summary
Pandas is fast if used right: use vectorization instead of Python loops or apply — on a 1M-row, 95 MB test the loop version takes ~12.5s vs 0.08s vectorized (~150×). Fix dtypes with convert to categorical and downcast numerics to cut memory and speed joins/groupbys; avoid chained ops to prevent hidden copies and prefer boolean masking with .loc. Use read_csv(usecols, dtype), measure with %timeit and df.info, and consider Polars and DuckDB or Parquet/Feather for parallelism, lazy eval, and columnar IO.
AI-written summary · about
chapters (9)
- 0:00Faster pandas intro
- 1:11Why pandas slows down
- 9:14Slow pandas mistakes
- 13:31Fixing data types
- 17:24Avoiding hidden copies
- 21:55Optimizing joins and group by
- 24:47Better file formats than CSV
- 27:19Pandas alternatives
- 32:18Fast pandas takeaways
related talks
44mHigh Perfomance Pandas
32mPandas May Be Slow, but Pandas Doesn't Have to Be
31mFaster Pandas: Speed Up Your Code, Shrink Your Cloud Bill
35mIdiomatic Pandas
29mMaking Pandas Fly
31mBeyond Pandas: lightning fast in-memory dataframes with Polars
28mIdiomatic Pandas
01h 17mPandas from the Inside / "Big Pandas"