Terraforming Jupyter and Dask: How to Get Your Own Cloud Data Science Platform on the Cheap

on YouTube ↗26m

" summary

Open-source platform QHub deploys multi-tenant JupyterHub plus Dask on Kubernetes across AWS, GCP, DigitalOcean and on‑prem using Terraform, Helm and GitHub Actions; a single ~50-line YAML drives CI that runs terraform plan/apply and pushes container images. QHub launches notebook servers and autoscaling Dask workers on demand, supports GPU node groups, and runs local tests with minikube. QHub deploys in under 15 minutes and Terraform 0.13 fixes module dependency pain.

AI-written summary · about

" chapters (6)

1. 0:00Introduction and motivation
2. 3:49Python data science ecosystem
3. 7:54QHub platform overview
4. 10:50Demo of Jupyter and Dask
5. 14:02Terraform for infrastructure
6. 19:47Summary and Q&A

" related talks

• 27mDeploying JupyterHub with KubernetesCamilla Montonen · PyData Amsterdam 2018
• 29mLaunch Jupyter to the Cloud: an example of using Docker and TerraformCheuk Ting Ho · PyData Berlin 2018
• 39mHow We Hosted 200+ Data Scientists in a K8S ClusterAleksey Kuznetsov, Mikhail Lepeshkin · Saint HighLoad++ 2022
• 41mDeploying DaskMatthew Rocklin · PyData NYC 2022
• 01h 15mEasily deploy+maintain an open source data science platform on the cloud of your choices Guild · PyData NYC 2022
• 44mEmpowering Data Science with Kubernetes | Samuele Chiocca and Andrea RotaSamuele Chiocca, Andrea Rota · containerday 2020
• 28mDeploying DaskMatthew Rocklin · PyData Global 2022
• 35mKubeflow: Data Scientist ready?Sylvain Lequeux · DevFest Paris 2019