A practical guide to analysis & interactive visualization of massive datasets
summary
Hands-on walk-through uses a Nebari cloud lab with 50 instances to analyze the US DOT Airline On‑Time Performance 70 GB dataset stored as Parquet, showing Dask dataframes with lazy evaluation, partitioning, shuffling and when to persist intermediate results. It provisions clusters via Dask Gateway (examples use 5–10 workers), visualizes with HVPlot/HoloViz and DataShader, merges airport lat/longs to map delays, and demonstrates that operations can take ~1–2 minutes; remember to shut down clusters to avoid cloud charges.
AI-written summary · about
chapters (12)
- 0:00Massive dataset analysis intro
- 10:29Hands-on environment setup
- 24:43Dask for parallel computing
- 39:02Dask task graphs & lazy eval
- 51:06Dask clusters & distributed computing
- 1:04:50Dask workflow best practices
- 1:14:25HoloViz & HvPlot intro
- 1:22:43Interactive HvPlot visualizations
- 1:31:22HoloViz Explorer data exploration
- 1:35:20HvPlot geographic plotting
- 1:39:27DataShader for massive datasets
- 1:43:05Q&A
related talks
01h 25mA practical guide to analysis and interactive visualization of massive datasets
03h 07mData of Unusual Size: Interactive Visualization
40mData-centric exploration using intake, dask, hvplot, and more
53mTips and tricks for robust big data applications
28mProcessing Large Geospatial Datasets with Dask & Xarray
26mWho Said Wrangling Geospatial Data at Scale was Easy?
22mTake your Business to a New Geography
36mMaking interactive visualizations easy, inside and outside of notebooks