Jupyter extension for executing kubeflow pipeline seamlessly
summary
Elyra ties Jupyter to Kubeflow Pipelines v2 so notebooks become pipeline components via a visual drag‑and‑drop editor and one‑click pipeline submission. The extension uses papermill to reconstruct pipelines into the new v2 pipeline spec, targets Argo on Kubernetes, stores artifacts in S3, and can export Python DSL or YAML for direct submission. Demo runs on Open Data Hub / OpenShift, supports custom runtime images and GPUs, and shows pipelines executing with metadata and logs preserved.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:06AIML lifecycle focus
- 3:57Kubeflow pipelines for parallel workflows
- 5:02What is Kubeflow
- 6:27Jupyter extension Alra
- 8:12Alra with Jupyter and Kubeflow
- 10:20Kubeflow V2 updates and Alra
- 13:10Demo setup
- 16:22Jupyter workbench setup
- 17:44Building pipelines in Alra
- 20:13Executing pipelines
- 22:20Key features
- 22:41References
related talks
42mBuilding ML Pipelines in JupyterLab Using Elyra - Without the Need to Write Code
26mStreamlining AI Pipelines With Elyra: From Development To Inference With KServe & VLLM
31mKubeflow - Data Science on Steroids
32mA Deep Dive into Kubeflow Pipelines
36mMachine learning pipeline Kubeflow, let's start
35mOrganizing data engineering around Kubeflow
29mML Engineering Kubeflow
25mScaling ML pipelines with KALE — the Kubeflow Automated Pipeline Engine