Building ML Pipelines in JupyterLab Using Elyra - Without the Need to Write Code
Talkat Open AI + Data Forum @ Open Source Summit North America 2022 · on Tuesday, 2022-06-21· at 14:05· Room 408/409 (Level 4), JW Marriott Austin, 110 E 2nd Street Austin, TX 78701·
summary
Elyra extends JupyterLab with a visual pipeline editor that wires notebooks, Python/R scripts and custom components into runnable workflows, compiles them to Kubeflow Pipelines or Airflow, and submits runs to remote clusters or a local runtime. Users specify a container image for repeatable environments, GPU and resource requests, data volumes for large file exchange, and secrets; artifacts land in object storage. Project averages monthly minor releases, ~50 contributors, and components are production-ready after six months of work.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 2:00What is Elyra
- 6:22Motivation
- 12:41Pipeline Editor
- 15:07Pipeline Properties
- 21:37When to use components
- 23:57How to implement components
- 27:51Running a pipeline
- 28:51Runtime Environments
- 29:30Running the Pipeline
- 32:18Viewing the Pipeline Results
- 32:42Running the Pipeline Remotely
- 34:10Deployment Options
- 35:42Useful Links
- 36:32Questions
related talks
27mJupyter extension for executing kubeflow pipeline seamlessly
26mStreamlining AI Pipelines With Elyra: From Development To Inference With KServe & VLLM
25mScaling ML pipelines with KALE — the Kubeflow Automated Pipeline Engine
36mThe Lifecycle of a Jupyter Environment
29mML Engineering Kubeflow
05mLightning Talk: Build a Reproducible ML Workflow with Kubeflow Pipelines
36mMachine learning pipeline Kubeflow, let's start
35mUntitled12.ipynb