Reproducibility in ML Systems: A Netflix Original
summary
Claims that reproducibility is a first-class ML-infrastructure goal and shows an actual system that enforces it: per-run code packaging to S3 with versioned run artifacts, Conda-based runtime isolation so steps can use conflicting libraries, artifact hashing to avoid duplicates, and a resume feature that reruns from any prior step. Concrete tradeoffs include not storing full gigabyte-scale raw datasets by default and accepting limited entropy capture (seeds only), plus using persisted metadata and logs to make debugging and on-boarding practical.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:03ML use cases at Netflix
- 3:11ML system components
- 6:04Defining reproducibility
- 7:09Poor reproducibility effects
- 11:30Why reproducibility matters
- 14:43How Netflix addressed reproducibility
- 23:57Metaflow benefits
- 25:49Q&A
related talks
30mReproducible Machine Learning
32mReproducibility and versioning of ML systems
30mReproducible Machine Learning at Scale
33mBuilding Reproducible ML Processes with an Open Source Stack
30mHuman-Centric ML Infrastructure at Netflix
01h 16mReproducible machine learning and science with python
30mRealizing End to End Reproducible Machine Learning on Kubernetes
20mBetter Care and Feeding of Machine Learning Models