Learning to Scale Data Science, Machine Learning, and Pandas with Ray and Modin
summary
A unified runtime called Ray actually runs Ray remote tasks and actors with a bottom-up scheduler, per-node object store, Redis-backed global control store, and GPU support, and a data library called Modin accelerates Pandas by a single import so Modin one-line change scales to clusters. Claims include supporting 71% of the pandas API (93% of observed usage), defaulting to pandas for rare APIs, a cloud demo and exercises, and real use cases that cut simulator training from about eight hours to minutes.
AI-written summary · about
chapters (12)
- 0:00Ray and Modin intro
- 1:58Distributed ML challenges
- 3:45Ray tasks and actors
- 5:09Remote functions in Ray
- 7:12Ray actors for stateful
- 8:15Ray architecture and fault
- 12:10Modin scales pandas
- 13:41Modin’s API implementation
- 15:57Unsupported pandas ops
- 17:02Modin adoption status
- 23:11ML workloads impact
- 25:22Ray and Modin tutorial
related talks
21mScaling Interactive Data Science with Modin and Ray
24mData Scientists are More Valuable than Hardware: The Design of Modin
01h 31mIntroduction to Ray for distributed and machine learning applications in Python
24mRay: A System for High-performance, Distributed ML Applications
57mIntroduction to Ray for distributed and machine learning applications in Python
26mRay: A System for High-performance, Distributed Python Applications
22mRay, a Unified Distributed Framework for the Modern AI Stack
15mRay: A System for Distributed Applications