Reproducibility and versioning of ML systems
summary
Explains exactly what must be versioned to reproduce results: dataset id and timestamp, train/test split and shuffling, preprocessing and feature transforms, model architecture and hyperparameters, source code and floating‑point environment. Recommends concrete tools: git + DVC for data, experiment tracking with MLflow or Neptune, Docker for environment images, model registry for serialized models, Kubeflow/SageMaker for pipelines, and a central metadata store to link versions. Claims these pieces enable rollbacks, point‑in‑time feature lookups, and auditable production deployments.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:26Defining reproducibility
- 3:13Why reproducibility matters
- 5:40Benefits of modular ML code
- 7:06Key parameters to track
- 9:14Git and ML versioning tools
- 10:16Experiment tracking
- 11:21Dataset versioning with DVC
- 12:03Feature stores
- 14:57Model versioning
- 17:01Environment versioning with Docker
- 18:28Workflow orchestration
- 19:55Infrastructure as code
- 20:36Metadata storage
- 21:39How components fit together
- 24:46Documentation best practices
- 25:08Q&A: ML versioning tools
- 26:54Q&A: Environment setup
- 29:25Q&A: Metadata storage
related talks
24mModel Versioning: Why, When, and How
59mDevOps for Machine Learning Projects
29mReproducibility in ML Systems: A Netflix Original
33mBuilding Reproducible ML Processes with an Open Source Stack
01h 25mBest practices and solution design using data versioning for machine learning
37mOpen Source Tools for ML Experiments Management
29mML Model Traceability and Reproducibility by Design
32mMachine Learning Models Versioning Using Open Source Tools