Productionizing H2O Models with Apache Spark
summary
A concrete workflow shows H2O models embedded inside Apache Spark so a single serialized artifact handles training, feature transforms and serving; it uses Spark pipelines with Sparkling Water and produces an H2O MOJO pipeline (Driverless AI also exports MOJOs) that runs on streaming S3. Demo timings are specific: a model train stage demo set to 60 seconds and production init takes about half a minute to initialize. The pipeline runs and serves predictions end to end.
AI-written summary · about
chapters (11)
- 0:00H2O and Spark integration
- 3:59Sparkling Water overview
- 7:17Spark pipeline with H2O models
- 12:42Driverless AI in Spark pipelines
- 18:08Productionizing with Spark Streaming
- 21:22Extending ML lifecycle
- 24:37Summary and resources
- 25:01Q&A: Spark ML vs H2O
- 26:24Q&A: H2O deep learning licensing
- 28:39Q&A: Driverless AI transparency
- 29:46Q&A: Model management
related talks
32mDifferent Strategies of Scaling H2O Machine Learning on Apache Spark
27mDeploying distributed AI and machine learning in financial services
27mDeveloping and Deploying Models at Scale — without Spark!
50mScalable Data Science & Deep Learning with H2O
01h 10mGetting started with H2O on Python
45mMassive distributed processing with H2O
37mBig Data Analytics, Machine Learning, Real Time Processing
01h 18mIntroduction to H20 AutoML with Python