How Narwhals is silently bringing pandas, Polars, DuckDB, PyArrow, and more together
summary
Narwhals provides a lightweight compatibility layer that lets libraries use pandas, Polars, PyArrow, DuckDB and more without adding dependencies, claiming zero required dependencies and full static typing; Plotly’s switch to Narwhals yields measured speedups (Polars users see up to 10x in examples) because backends do the compute and Narwhals only dispatches. It exposes a Polars-like expression API with from_native → to_native wrappers, preserves eager vs lazy semantics (explicit ordering required for cumulative sums), and the project has 100+ contributors and a stable namespace for long-term compatibility.
AI-written summary · about
chapters (12)
- 0:00Introduction to Narwhals
- 1:22Roadmap and goals
- 2:37The problem with dependencies
- 4:12The Narwhals solution
- 5:48Design philosophy
- 7:58Crash course on usage
- 10:43Eager vs lazy execution
- 12:27Lazy cumulative sums
- 13:40Live coding demonstration
- 15:38Community and future
- 17:01Conclusion
- 17:51Q&A and closing
related talks
34mHow you can write a dataframe-agnostic library
32mPandas, Polars and the DataFrame Consortium.
30mIs the great dataframe showdown finally over? Enter: polars
30mPandas 2, Dask or Polars? Tackling larger data on a single machine
17mMaking Things Nice in Python
58mFrom Pandas to Polars Upgrading Your Data Workflow
15mData Refinement: Unleashing the Power of SQL in Python using Polars
31mPolars Plugins: how you (yes, you!) can extend Polars Dataframes