Density! The not-so-secret trend driving the future of Data Analytics
summary
Argues that densifying hardware — packing more CPU cores, H100 80 GB GPUs, higher memory bandwidth and NVLink — changes what problems are practical: reproducing a large LLM trained on 15 trillion tokens and ~30 million GPU hours becomes about memory locality and coherence not raw FLOPS. Shows cloud CPUs/GPUs now available as dense single instances, cites tools that must adapt (NumPy/Pandas, RAPIDS, Dask, Ray, CUDA), describes multi‑GPU rigs (8+ GPUs, NVLink islands, racks of 72 GPUs) and reports that keeping data on-device materially speeds common analytics.
AI-written summary · about
chapters (21)
- 0:00Computational density introduction
- 3:13Hardware-software coevolution
- 6:04Ancient data records
- 7:081960s parallelism
- 9:381960s software unification
- 11:21Connection Machine computing
- 13:07Vectorized APIs and Fortran
- 16:39Modern architecture and NumPy
- 18:23PC revolution and data explosion
- 20:06MapReduce and commodity hardware
- 22:10Pandas and NumFocus rise
- 23:59Distributed programming challenges
- 25:46Model reproduction thought experiment
- 27:12Denser hardware shift
- 27:54NVIDIA AI solutions and GPUs
- 30:44Coherent memory paradigms
- 32:11Scaling to 72 GPUs and 36 CPUs
- 33:15Rapids APIs for analytics
- 34:38Zero-code GPU usability
- 36:28Community tool development
- 37:33Dense machines era
