End-to-End Computation on the GPU with a GPU Data Frame
summary
MapD runs SQL, visualization and model prep entirely on GPUs and demonstrates end-to-end zero-copy IPC between processes using Apache Arrow as the on‑GPU columnar exchange. Benchmarks show single‑server queries of a 1.2 billion row taxi set in 20–50 ms on one server and server‑side rendered maps of an 11.6 billion row ships dataset, plus a Python import improvement to about 42 ms for ~3 million rows, enabling interactive feature engineering and ML without round trips to CPU.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 4:14CPU vs GPU
- 4:54MathD
- 7:10Demo
- 10:55Ecosystem
- 12:41Speed
- 13:29GPU Data Frame
- 15:18Use Cases
- 17:20Zero Copy Framework
- 18:48Python Demo
- 20:17Question
- 21:03Get involved
- 21:25Memory caching
related talks
23mAccelerate the Machine Learning Pipeline on Very Large Datasets
32mVectorized Query Processing for CPUs using Apache Arrow
14mFrom Reservation to 1,600 GPUs: A Petabyte-Scale Ray Data Pipeline
32mAnyscale's Ray Data: Revolutionizing Batch Inference
18mData Preprocessing for ML at Production Scale
33mArrow in Flight - New Developments in Data Connectivity
05h 34mData @Scale 2024
42mMulti-GPU Accelerated Processing of Time-Series Data of Huge Academic