Data Science at Scale with Dask
summary
Dask lets familiar PyData APIs scale: pandas/numpy/scikit-learn become distributed collections, and arbitrary Python gets parallelized via dask.delayed and futures. Demo shows local multicore development, then the same code on a cloud cluster with dashboards and 20 workers via coiled. A Capital One case moves a pipeline from 2.5 weeks → 15 hours by using 80 workers ~700 GB of distributed memory. Lazy evaluation means you call compute or persist to execute.
AI-written summary · about
chapters (15)
- 0:00Start
- 3:05What Is Dasc
- 7:39Das Dashboard
- 19:13Lazy Evaluation
- 22:47Das Arrays
- 27:13Is Dax Ml Only Usable on a Local Machine or Also on a Compute Cluster
- 32:13Scale to the Cloud
- 42:28When To Use Dusk
- 43:57Das Data Frame Api
- 53:11Distributed Scheduler
- 59:33What Is the Best Strategy for Data Location
- 1:00:56The Difference between Dax Data Frames and Dax Bags
- 1:05:07Single Machine Parallelism
- 1:12:13Memory Bound
- 1:21:24Can You Share Jupiter Notebook
related talks
01h 24mData Science at Scale with Dask
01h 17mScaling PyData with Dask
01h 03mDask: Next Steps in Parallel Python
01h 22mUsing Dask for Parallel Computing in Python
01h 56mParallelizing Scientific Python with Dask
01h 26mDask tutorial
38mScalable Machine Learning with Dask
23mData Science at Scale Using dask and Numba