How to easily set up and version control your Machine Learning Pipelines
summary
Promises concrete setup and shows it actually working: step‑by‑step project initialized with DVC and MLV-tools, data and intermediate files versioned via DVC cache and remote storage, pipelines declared as DVC stages and reproduced with dvc repro, notebooks converted to executable scripts with epynb-to-python, and experiments compared by branching under git‑backed experiments. Demonstrates docker-compose sandbox, common failure modes (outdated metadata, tedious manual deps), and a working flow that automates notebook→script→versioned pipeline.
AI-written summary · about
chapters (18)
- 0:00ML pipeline versioning intro
- 3:27Why ML pipelines need version control
- 7:06DVC for pipeline versioning
- 9:38Tutorial setup overview
- 11:35Setup DVC and Git repo
- 22:32Build simple pipeline with DVC
- 31:33Track pipeline steps
- 40:20Reproduce experiments with DVC
- 47:06Advanced DVC features
- 51:37Remote storage for collaboration
- 53:24MLV tools for DVC
- 57:33Install MLV tools
- 1:00:08Convert notebooks to scripts
- 1:11:44Configure notebooks for MLV
- 1:22:06Generate scripts and DVC commands
- 1:25:56Run ML pipeline end-to-end
- 1:28:16Hyperparameter tuning with DVC
- 1:30:50Q&A
related talks
26mTransforming a Jupyter Notebook into a reproducible pipeline for ML experiments
16mMake your data science workflows reproducible
42mApplications in ML Drug Discovery pipelines
35mData versioning in machine learning projects
59mDevOps for Machine Learning Projects
02h 10mBuild End-To-End MLOps Platform with Open Source DVC Ecosystem
33mFlexible ML Experiment Tracking System for Python Coders with DVC and St...
37mOpen Source Tools for ML Experiments Management