From Jupyter Notebook to Eruption: Getting Geohazard Research into Production
summary
Earth Sciences New Zealand runs a platform that moves Jupyter notebooks from laptops into production via risk-tiered environments and a clear tier progression: commit to on‑prem GitLab, add a Dockerfile and CI/CD, expose a FastAPI endpoint using a small Python package, then deploy to Portainer or an on‑prem cluster and finally to GitHub + ECS with GitHub Actions. It supports >1,000 sensors in ~700 locations and tens of terabytes/year, uses bespoke Grafana plugins, and works — but ownership gaps, leaking tiers, silent decay and dependency rot remain.
AI-written summary · about
chapters (10)
- 0:00Geohazard research introduction
- 1:27New Zealand hazards overview
- 5:23Scientific code challenges
- 7:52Risk-tiered environments
- 9:35Jupyter to shared infra
- 13:07Tier two onboarding
- 14:17Tier three cloud deployment
- 18:57Research software takeaways
- 22:12Closing remarks
- 22:33Q&A
related talks
39mNotebook To Production
32mDeploying Python models to production
26mAutomating runbooks with Jupyrest
26mTerraforming Jupyter and Dask: How to Get Your Own Cloud Data Science Platform on the Cheap
36mBuilding a Scalable Data Science & Machine Learning cloud using Kubernetes
17mR & Python playing nice, in Production
29mLaunch Jupyter to the Cloud: an example of using Docker and Terraform
01h 36mDeploying Jupyter Notebooks for students and researchers