Understanding Polars data types
summary
Polars claims 18 data types and demonstrates concrete tradeoffs: bit-packed booleans save memory, a validity mask makes is_null free at compute cost of a bit per row, floats expose NaN versus null semantics, decimals give fixed higher precision, enums require a predefined set of values and raise errors on invalids, temporal columns enforce a single timezone per column, and List vs Array, Categorical vs Enum and Struct are distinguished by storage, semantics and available specialized operations that affect performance and memory.
AI-written summary · about
chapters (16)
- 0:00Polars data types intro
- 1:52Why Polars types matter
- 3:18Boolean type and memory
- 5:29Null type for missing
- 8:39Integer types in Polars
- 11:10Signed vs unsigned integers
- 14:50Floating-point precision
- 17:16String type and text
- 18:19Temporal types: date/time
- 20:08DateTime type and time zones
- 22:37String to temporal conversion
- 24:05Duration type for intervals
- 25:11Binary type for raw data
- 26:18Decimal type for precision
- 28:07Enum and categorical types
- 29:31Wrap-up and resources
related talks
01h 32mAn Introduction to Polars
28mExpressive & fast dataframes in Python with polars
38mExpressive and fast dataframes in Python with polars
58mFrom Pandas to Polars Upgrading Your Data Workflow
01h 16mPower up your Polars code with Polars extention
28mAdvanced Polars: Lazy Queries and Streaming Mode
30mIs the great dataframe showdown finally over? Enter: polars
36mHow Narwhals brings Polars, DuckDB, PyArrow, & pandas together