A billion stars in the Jupyter Notebook
summary
Vaex exposes columnar HDF5-backed tables and processes a billion rows per second by memory‑mapping and streaming chunked expressions to compute N‑dimensional binned statistics (counts, means, std, etc.) without materializing intermediates. ipyvolume delivers volume rendering and up to a million glyphs as a Jupyter widget, ties to ipywidgets/leaflet for linked zooming and selections, exports interactive HTML, and supports simple stereo VR with Google Cardboard; demos show Gaia billion‑star maps and NY taxi aggregates.
AI-written summary · about
chapters (17)
- 0:00Billion-star visualization in Jupyter
- 0:24Motivation for billion-star work
- 1:47Scatter plot challenges with big data
- 3:11Memory and performance limits
- 4:39HDF5 and memory mapping
- 6:24ND statistics with effects
- 8:12Data types in effects
- 10:21Demo: 2D stats in effects
- 14:37Remote billion-star datasets
- 16:02Interactive Ipywidgets
- 18:29Maps with Ipyleaflet
- 20:57Demo: Temporal spatial exploration
- 21:393D with Ipyvolume
- 24:48Ipyvolume features
- 26:35Demo: 3D stellar streams
- 29:05Demo: Interactive billion-star exploration
- 29:47Conclusion and key takeaways
related talks
43mAdvanced Data Visualizations In Jupyter Notebooks
46mExplorations in Interactive Visual Analytics
32mAdvanced Visualization in Jupyter Notebooks
37mKeynote – Carol Willing
29mUsing Jupyter notebooks to develop and share interactive data displays
36mMaking interactive visualizations easy, inside and outside of notebooks
32mLightning Talks
28mAdvanced Graph Visualizations in Jupyter Notebooks