Panel: NVIDIA GPU-accelerated Spark
summary
Panel argues GPUs bring orders-of-magnitude core counts to Spark workflows, speeding deep learning training and inference by running TensorFlow and JNI-backed JVM libs on DGX-1 and GPU instances with drivers and runtimes preinstalled. It shows concrete tools and integrations: TensorFrames on Spark DataFrames (aligned with tungsten and Spark 2.0) and JVM stacks like DL4J/ND4J, enabling transparent Spark ML acceleration and parallel hyperparameter tuning and large-scale inference/feature extraction that demonstrably reduces time to result.
AI-written summary · about
chapters (10)
- 0:00Introduction and setup
- 2:04Panelist introductions
- 6:43NVIDIA's role in GPU acceleration
- 9:54Spark and GPU integration benefits
- 13:01Deep learning and Spark workflows
- 15:09Audience Q&A begins
- 15:50Databricks and GPU support
- 17:37Project Tungsten and GPU progress
- 20:27SQL and computer vision challenges
- 22:07Closing remarks
