Accelerate the Machine Learning Pipeline on Very Large Datasets
summary
mapD is a columnar SQL database built for GPUs and claims about 100x speed versus CPUs by loading queried columns into GPU memory. A live demo runs four brute-force SQL queries over 13 billion rows in ~300 ms and renders a 630 million‑row map point-by-point; another demo queries ~380 million geolocated tweets. The stack uses py-mapd, a GPU DataFrame and Apache Arrow for zero-copy transfers; community edition runs as Docker/AMI and AWS GPU instances cost ~$0.90/hr.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 2:07Data is now eating the world
- 3:15MapD
- 5:33Shipping Data
- 8:18Twitter Data
- 11:28Memory Management
- 13:15GPU Open Analytics Initiative
- 17:51New thinking about the ML Pipeline
- 21:37Next steps
related talks
24mEnd-to-End Computation on the GPU with a GPU Data Frame
39mAccelerating Data Science with RAPIDS
41mAccelerating Data Science with RAPIDS
01h 22mHigh-Performance Data Science at Scale with RAPIDS, Dask, and GPUs
32mVectorized Query Processing for CPUs using Apache Arrow
06mMachines and the magic of fast learning
01h 08mGPU Data Science with Rapids
05mDriving the On-Demand Economy with Predictive Analytics