Impact on business and society
summary
Salesforce builds a multi-tenant ML platform called Optimus Prime on Scala and Spark that automatically ingests Service Cloud data, applies automated feature engineering, runs per-tenant model bake-offs (logistic regression, random forest, SVM via SparkML), selects and tunes winners with cross-validation and sampling, versions and deploys tenant-specific models, and monitors degradation. The stack uses AKKA and AWS lambda-style orchestration, supports classification and regression, and already serves hundreds of thousands of customers, reducing human effort with type-safe declarative syntax that prevents runtime schema errors.
AI-written summary · about
chapters (21)
- 0:00Start
- 1:41Artificial Intelligence in your everyday life
- 3:32Standard Methodology for building Al apps Cross Industry Standard Process for Data Mining CRISP-OMO
- 5:57How companies build ML apps
- 6:30The Problem
- 8:51Should we hire all the world's data scientists?
- 13:28Einstein Builds A Specific Model for Each Customer Per App
- 14:37Building a Platform that enables Automated Custom Modeling per Tenant
- 15:20The Salesforce Use Case
- 21:08AI 101
- 22:59Feature Definitions in SparkML
- 24:56Feature Definitions in Optimus Prime
- 26:01Feature Transformations in Spark ML
- 26:47Feature Transformations in Optimus Prime
- 28:13Tribal Knowledge in Spark ML
- 30:20OP Pipelines translate to Spark ML Pipelines
- 31:32Shared Feature Repository
- 32:33Automated Feature Engineering
- 35:26What Now?
- 35:58Automatic Model Selection and Hyper-parameter Tuning
- 38:31Key Takeaways
related talks
34mComplex Machine Learning Pipelines Made Easy
38mFantastic ML apps and how to build them
38mReal Time ML Pipelines in Multi-Tenant Environments
07mThomas Gerber Interview
42mThe Lego Model for Machine Learning
46mIntroducing Salesforce Genie: A real-time platform for customer magic
29mIntelligent Log Analysis and Real-time Anomaly Detection @ Salesforce
23mML Platform Challenges & Applications at Freshworks