Ameet Talwalkar, Assistant Professor of Computer Science, UCLA @ MLconf SF Part 2

on YouTube ↗16m

" summary

MLlib on Spark gets an automated hyperparameter search layer called MLbase optimizer that combines batch model updates (matrix‑matrix LAPACK routines), early stopping framed as a multi‑armed bandit, and smarter search (random search, TPE/SMAC). Batched linear algebra yields about a 5x speedup for many features, bandit pruning yields another ~5x with no loss in validation accuracy, and the combined system reports roughly a 20x speedup on a 30 GB problem and finds a good model on a >1 TB dataset in ~11 hours on 128 nodes.

AI-written summary · about

" chapters (11)

1. 0:00Training A Model
2. 0:34The Tricky Part
3. 1:11A Standard ML Pipeline
4. 1:23One Approach
5. 2:15A Better Approach
6. 3:02A Tale Of 3 Optimizations
7. 3:45What Do We See In Spark?
8. 5:02Bandit Search
9. 8:18Putting It All Together
10. 9:06Does It Scale?
11. 10:27Other Future Work

" related talks

• 39mMachine Learning with Scala on SparkJose Quesada · Scala Days Berlin 2016
• 38mFantastic ML apps and how to build themMatthew Tovbin · Scale By the Bay 2017
• 26mMLbase: A Distributed Machine Learning SystemTim Kraska · Parallel and Distributed Algorithms for Inference and Optimization (Simons Institute, Oct. 21–24, 2013)
• 44mApache Spark and ML workflowsJuliet Hougland · Jazoon Zurich Autumn 2017
• 01h 22mAutomated Machine Learning & Tuning with FLAMLJiang et al. · PyData Seattle 2023
• 17mMachine Learning at the Speed of Light with SparkLudwine Probst · Scala.IO 2014
• 16mAndrew Musselman, Committer & PMC Member, Apache Mahout at MLconf Seattle 2017Andrew Musselman · MLconf Seattle 2017
• 50mApache Spark vs Machine LearningPatryk Pilarski · Confitura 2019