~/talk

Is the great dataframe showdown finally over? Enter: polars

on YouTube 30m
Talkat PyCon Italia 2023 · 2023-05-25..05-28· [data-engineering, python]

speaker

name:Luca Baggi
links:none on file

resources

video:https://youtu.be/XsxrhGlkyk0
slides:none on file

summary

Polars claims multi threaded, Arrow based execution with a query planner that does predicate pushdown and lazy scan to lazy frame streaming so users can process datasets like 200 GB on a laptop by reading parquet chunks and syncing results to a single parquet. It converts pandas cheaply via Arrow but converting an in‑memory pandas DF to lazy does not reduce peak memory. Groupbys, window functions and profiling are native and fast; UDFs and scan‑SQL for databases remain limited.

AI-written summary · about

chapters (7)

  1. 0:00Introduction and motivation
  2. 2:58Pandas 2.0 vs. Polars
  3. 6:24Polars data ingestion
  4. 10:05Syntax and data manipulation
  5. 15:38Lazy evaluation and scaling
  6. 18:40SQL support and command line
  7. 22:03Q&A session

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.