Streamlining AI Pipelines With Elyra: From Development To Inference With KServe & VLLM
summary
Elyra converts Jupyter notebooks into executable Kubeflow DAGs so a data scientist drops in Python cells and Elyra converts notebooks to Kubeflow pipelines producing a visual DAG that runs on JupyterHub/Open Data Hub and stores artifacts in MinIO/S3. A separate Tekton pipeline watches the model registry and Tekton CI/CD triggers model deployment into OpenShift where KServe provides KServe autoscale to zero inference endpoints and vLLM speeds LLM serving and lowers GPU use with vLLM reduces GPU memory via page attention, examples and demo videos show pipelines and deployments working end-to-end.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:46Data scientist pipelines with Elyra
- 6:18Elyra visual editor & Jupyter
- 9:26RAG workflows with Elyra
- 12:55KServe for model deployment
- 15:58VLLM for LLM inference
- 18:12OpenShift AI UI
- 20:39MLOps & CI/CD pipelines
- 24:59Key takeaways
related talks
20mInferencing LLMs in production with Kubernetes and KubeFlow
42mBuilding ML Pipelines in JupyterLab Using Elyra - Without the Need to Write Code
27mJupyter extension for executing kubeflow pipeline seamlessly
49mTutorial: From Notebook to Kubeflow Pipelines to KFServing: the Data Science Odyssey
17mDevOps for Private AI: A Scalable Platform Using Kubeflow
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
54mScaling Machine Learning Pipelines in Cloud
27mKubeflow for enabling AI Powered Drug Discovery and development...