TransmogrifAI - Automate ML Workflow with power of Scala and Spark at massive scale.

on YouTube ↗48m

" summary

TransmogrifAI is an open-source Scala library built on SparkML for end-to-end automated machine learning on heterogeneous structured data. It provides compile-time type safety and modular components for feature inference and transformation, automated feature validation, model selection and hyperparameter optimization to shorten development and deployment time. The talk covers using TransmogrifAI in Scala/Spark applications and presents use cases and integration details.

AI-written summary · about

" chapters (19)

1. 0:00Start
2. 2:04What is TransmogrifAl?
3. 11:27Under the Hood
4. 12:31Automated Feature Engineering
5. 16:25Automated Feature Selection / Data Pre-processing
6. 17:57Auto Data Pre-processing
7. 19:05Auto Selection of Best Model with Hyper Parameter Tuning
8. 19:57Examples - Hyper parameter tuning
9. 21:24Ensemble Modeling
10. 22:31Structured Data in Apache Spark
11. 25:24DataFrame API Code
12. 25:59DataFrame- SQL View - SQL Query
13. 27:12Catalyst in Spark
14. 30:13Spark SQL API - Analysis Error example.
15. 31:50Example Code
16. 42:42Apache Livy Configuration
17. 43:18Apache Livy Integration ...
18. 44:39Questions?
19. 47:45PLATINUM SPONSOR

" related talks

• 39mMachine Learning with Scala on SparkJose Quesada · Scala Days Berlin 2016
• 38mFantastic ML apps and how to build themMatthew Tovbin · Scale By the Bay 2017
• 23mRethinking scalable machine learning with Spark ecosystemAdi Polak · Scale By the Bay 2021
• 34mComplex Machine Learning Pipelines Made EasyChris Rupley, Till Bergmann · Scale By the Bay 2017
• 47mAutoML on Spark: The Myth That Became RealityNikolay Butakov, Aleksandr Ryzhkov · Joker 2023
• 35mPySpark : Combining Machine Learning & Big DataAyon Roy · Data Con LA 2020
• 38mEnabling Big Data and Machine Learning for the Masses...Tim Kral · Scale By The Bay 2018
• 04mInterviewVlad Patryshev · Scale By The Bay 2018