Using Reproducible Experiments To Create Better Models
summary
Demonstrates using DVC to make hyperparameter sweeps reproducible by versioning code, data and params and running queued experiments with DVC exp run against a small MNIST-style classifier that varies n_estimators and min_samples_split from a params.yaml file. Shows grid search (exhaustive combinations) versus random search and reports that random search finds comparable average precision and ROC AUC in fewer trials. Pipeline avoids re-running upstream stages when only train params change, supports cloud execution and produces plots and experiment IDs for exact reproduction.
AI-written summary · about
chapters (14)
- 0:00Introduction to talk
- 1:48Background on speaker
- 2:51What are hyperparameters
- 5:20Why reproducibility matters
- 8:58DVC for experiment tracking
- 11:25Grid search demo setup
- 15:17Grid search explained
- 17:22Running grid search experiments
- 19:48Analyzing experiment results
- 22:21Random search introduction
- 24:09Random search demo
- 26:58Benefits of reproducibility
- 29:08Key takeaways and wrap-up
- 31:42Live Q&A session
related talks
31mTuning Hyperparameters with DVC Experiments
16mMake your data science workflows reproducible
35mData versioning in machine learning projects
33mFlexible ML Experiment Tracking System for Python Coders with DVC and St...
46mFlexible ML Experiment Tracking System for Python Coders with DVC and Streamlit
27mLeveraging Git for ML Experiment Management
01h 32mHow to easily set up and version control your Machine Learning Pipelines
26mTransforming a Jupyter Notebook into a reproducible pipeline for ML experiments