Improve Your ML Projects: Embrace Reproducibility and Production...
summary
Kedro is a Python framework that turns messy notebooks into reproducible, production-ready pipelines by providing a project template, a centralized data catalog, composable nodes and pipelines, and experiment tracking that automatically logs parameters, metrics and artifacts. A live demo shows Kedro-Viz plotting metric changes across runs. Connectors exist for S3, GCS and HDFS, it outputs configurable configs and Docker-ready setups, and it supports separate inference pipelines that actually run.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:52What is Kedro
- 3:56Kedro as pipeline
- 5:42Project template
- 7:30Configuring environments
- 8:12Data catalog
- 11:44Understanding pipelines
- 13:34How nodes work
- 14:17Experiment tracking
- 15:21Visualizing metrics
- 16:26Extensibility and connectors
- 17:08Kedro for teams
- 18:56Conclusion and key takeaways
- 19:58Q&A
related talks
49mKedro + MLflow - Reproducible and Versioned data pipelines at scale
33mHow to build production-ready data science pipelines with
26mReproducible and maintainable data science code with Kedro
35mOrganizing data engineering around Kubeflow
40mData Science Best Practices with Kedro
33mCompliant Data Management and Machine Learning on Kubernetes
05mLightning Talk: Build a Reproducible ML Workflow with Kubeflow Pipelines
25mScaling ML pipelines with KALE — the Kubeflow Automated Pipeline Engine