Integrating Jupyter Notebooks into your Infrastructure
summary
A custom JupyterHub setup integrates Jupyter Notebook servers with existing LDAP and storage services and runs one isolated container per user using Docker images based on CentOS 7. It mounts central user homes over NFS so files are available everywhere, uses DockerSpawner and a Swarm spawner to distribute containers across nodes, and runs Xvfb for visualization with GPUs handled separately. Infrastructure scale mentioned is ~500 active staff, 10 compute clusters and ~4,500 cores and the demo shows the system working. LDAP authentication NFS-based storage Docker Swarm
AI-written summary · about
chapters (22)
- 0:00Intro
- 1:40Jupyter Notebooks
- 2:26Why use Jupyter Hub
- 3:20Getting started
- 4:24Central Storage
- 5:46ABAP Authenticator
- 6:52Docker
- 8:20Mounting volumes
- 9:47No X Server
- 12:20Spawner
- 14:47Kubernetes
- 15:35Authenticators
- 16:58Summary
- 18:30Tupolev
- 19:07Dynamic scaling
- 22:10Limit Research Usage
- 23:54What happens if the user doesnt properly sound
- 24:49Remove containers
- 25:47Questions
- 26:54Scripting
- 29:48Version Control
- 30:48Multiple Languages
related talks
01h 36mDeploying Jupyter Notebooks for students and researchers
31mBuilding an interactive training environment using JupyterHub
28mHosting Notebooks for 100,000 Users
40mRunning jupyter notebook remotely in a docker swarm cluster
27mDeploying JupyterHub with Kubernetes
36mHow to Roll your own Dockerized Jupyter Environment
36mBuilding a Scalable Data Science & Machine Learning cloud using Kubernetes
31mPanel: Jupyter in Education