Experimenting with Large Data Volumes Using Notebooks
summary
Shows how interactive notebooks (Zeppelin and Jupyter) can be integrated with Spark clusters in the cloud to run large-scale data experiments. Explains how this setup enables agile, efficient, and reproducible data-science projects through real-time visualization and easier experiment replication, and describes iFood’s practical use of this workflow to boost data-team productivity and business results.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:21What are notebooks
- 3:54Notebook architecture
- 6:02Exploratory data analysis
- 7:07Multimedia visualizations
- 11:45Large volume challenges
- 12:51Reproducibility and collaboration
- 18:14Spark and Data Lake integration
- 23:28Team collaboration
- 26:43Practical demo with Zeppelin
- 30:47Notebooks as new media
- 35:43Q&A
related talks
45mData Science Apps: Beyond Notebooks
50mBeyond Interactive: Empowering Everyone with Jupyter Notebooks
43mNotebook showdown: Jupyterhub vs. apache zeppelin
39mEnabling Data centric Teams
34mMoving notebooks into the cloud: challenges and lessons learned
06mLightning Talk: Using Jupyter Notebooks To Gain Insight Of Your Cluster
25mOvercoming 5 Hurdles to Using Jupyter Notebooks for Data Science, by the JetBrai...
42mApache Zeppelin