Beyond Interactive: Empowering Everyone with Jupyter Notebooks

on YouTube ↗50m

" summary

Netflix captures 1 trillion events/day into S3, processes mainly with Spark and serves results via Snowflake/Druid/Redshift/Elasticsearch; notebooks become an execution and orchestration layer rather than just a scientist sandbox. Built Interact front end, Papermill for parameterized, immutable runs, and Mason-triggered scheduling on Titus containers; this pipeline converts 10,000 jobs into notebook-driven ETL and runs 250,000 daily executions, improves debugging with bundled logs and enables engineers, analysts and PMs to publish interactive documentation and scheduled jobs.

AI-written summary · about

" chapters (44)

1. 0:00Intro
2. 0:30What is Netflix
3. 0:59Netflix international expansion
4. 1:28Netflix traffic
5. 3:55Company Culture
6. 4:25Breaking Bad Example
7. 6:28Data Platform Overview
8. 8:33Data Platform Users
9. 9:29Business Decisions
10. 10:40Personalization
11. 13:08Bandwidth
12. 14:17Flux
13. 14:57Data Engineers
14. 18:30Jupyter Notebooks
15. 19:27Interact
16. 21:14Login
17. 22:23Data Engineering
18. 23:41Scheduling
19. 24:17Triggering Notebooks
20. 25:33Notebook Template
21. 27:05Publish Event
22. 28:05Data Engineers Start Debugging
23. 28:34Data Quality Checks
24. 30:51Data Engineering Notebooks
25. 32:12Experimentation Solution
26. 34:54Questions
27. 35:24QA
28. 36:33Version Control
29. 37:37Connecting Data Analysts
30. 38:37What is the role of Tableau
31. 39:24What is the size of the data organization at Netflix
32. 40:16How do you promote reuse across notebooks
33. 41:05Is Jupyter located in Budapest
34. 41:24How do you fight against the information bubble
35. 42:08How do you govern data users and access
36. 43:36How do you protect against your enthusiastic graduate BA
37. 44:00How do you maintain the notebook environments
38. 44:39Do you prefer Presto over Spark
39. 45:04Managing knowledge coming from notebooks
40. 45:17Sharing notebooks
41. 46:06Data quality
42. 48:22AWS EMR
43. 48:44Communication
44. 49:06Skills Needed

" related talks

• 45mData Science Apps: Beyond NotebooksNatalino Busa · Codemotion Amsterdam 2017
• 49mExperimenting with Large Data Volumes Using NotebooksGilmar Souza · QCon São Paulo 2018
• 24mJupyter Notebooks for OpsDerek Arnold · LISA19
• 06mLightning Talk: Using Jupyter Notebooks To Gain Insight Of Your ClusterRuben D Orduz · KubeCon + CloudNativeCon Europe 2019
• 37mReproducible and shareable notebooks across a data science teamPascal Godbillot, Mike Tapi Nzali · Berlin Buzzwords 2022
• 01h 00mInteractive Computing with Jupyter: Past, Present, and FutureJason Grout · Philly ETE 2016
• 33mSolving the Scala Notebook ExperienceJeremy Smith, Jonathan Indig · Scale By The Bay 2019
• 25mTo Notebook or Not to Notebook: Multilingual Workflows for Data AnalysisRodrigo Silva Ferreira · PyCascades 2026