Fantastic ML apps and how to build them
summary
Salesforce builds an AutoML stack called Optimus Prime that uses Scala + Spark to automate data cleaning, vectorize feature generation, automatic feature selection, model selection and hyperparameter tuning, and evaluation; many models train hands-free and deploy at scale, serving about 0.5B predictions/day. A demo bootstraps a Titanic pipeline in minutes, selects a Random Forest with AUC 0.85 out of many candidates, and the team runs an open-source pilot to share the framework.
AI-written summary · about
chapters (12)
- 0:00Introduction and motivation
- 3:54Complexity in ML app development
- 8:36Functional programming benefits
- 11:10Why Scala for ML
- 13:19Type safety in feature engineering
- 17:10Automating vectorization
- 20:03Automatic feature selection
- 21:51Model selection strategies
- 22:56Demo: Titanic dataset
- 32:11Scaling predictions at Salesforce
- 33:17Key takeaways and resources
- 34:22Q&A session
related talks
49mImpact on business and society
39mMachine Learning with Scala on Spark
42mThe Lego Model for Machine Learning
31mUse AutoML to create high-quality models with just a few clicks
52mGenerate ML predictions without writing any code
31mUse-Case Driven ML Platforms
30mBring AutoML to all users across your business
21mBuilding Scalable ML Systems: Strategies and Insights