Notebook To Production
summary
Claims a reproducible path from notebook to production using six principles: host code as Python modules, version everything (code data models experiments), require peer review, track experiments, use task-oriented notebooks, and record environments. Uses Jupyter plus Git, nbdime, MLflow (remote server), large-file/versioning tools (DVC-style), Docker, GitHub Actions and DagsHub integrations. DagsHub provides notebook diffing and cell comments, a remote MLflow server and a GitHub-sync agent. Outcome is repeatable runs, simpler reviews, and production-ready scripts instead of fragile notebook-only deployments.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 2:45Why notebooks to production
- 4:46Jupyter strengths and flaws
- 7:13Six reproducible notebooks principles
- 8:15Moving code to modules
- 11:06Versioning notebooks and artifacts
- 13:36Versioning data and models
- 14:59Peer review for notebooks
- 16:24Tracking experiments and parameters
- 17:50Scaling collaboration with notebooks
- 19:34Reproducibility and environment tracking
- 21:24Six key principles recap
- 23:56Q&A
- 24:17Unit testing notebooks
- 25:21Running notebooks in production
- 27:09Dags Hub with Databricks
- 28:56Notebooks and IDE limitations
- 31:03Reusing functions across notebooks
- 32:51GitHub Actions with Dags Hub
- 35:00Jupyter notebooks evolution
- 37:53Closing remarks
related talks
26mTransforming a Jupyter Notebook into a reproducible pipeline for ML experiments
05mProductionalizing Data Science Models
35mGetting Machine Learning Models Ready For Production Using Python
01h 18mProductionizing Jupyter Notebooks
22mDock Your Jupyter Notebook
01h 25mTroubleshooting your Data Workflows with Noteable+Dagster-live debugging of failed jobs
28mFrom Jupyter Notebook to Eruption: Getting Geohazard Research into Production
01h 32mHow to easily set up and version control your Machine Learning Pipelines