Habitat: A Runtime-Based Computational Performance Predictor...
summary
Habitat predicts a single training-iteration execution time by profiling operations on an available GPU and scaling per-kernel timings to other GPUs, using wave scaling for kernel waves and pre-trained multilayer perceptrons for kernel-varying cases. Implemented as a Python library that supports PyTorch and is open source, it reports an average error of 11.8% average error across ResNet-50, Inception v3, Transformer, GNMT, and DCGAN on six GPUs (e.g., 2070, V100, T4), and it drives cost-versus-performance choices such as when V100 vs T4 is worth renting.
AI-written summary · about
chapters (11)
- 0:00Intro
- 0:26What this talk is about The problem: • Many GPUs available for deep neural network (DNN) training . Each has a different cost and performance
- 1:38A Cambrian explosion in hardware for training
- 2:23Choosing a GPU: The paradox of choice
- 4:22Key observations • Deep learning users may already have an existing GPU
- 5:03Habitat: A runtime-based performance predictor
- 10:24One last wrinkle: Kernel-varying operations Wave scaling assumes the same kernel is used across GPUS
- 10:45Evaluation
- 11:38How accurate is Habitat?
- 12:42Rent a GPU in the cloud? Scenario: Want to train GNMT, have access to a P4000. Which cloud GPU to use, if any?
- 15:09Key takeaways . DNN computation is special (repetitive), enabling new analysis opportunities
related talks
16mTowards GPU Utilization Prediction for Cloud Deep Learning
45mHigh-Performance and Scalable Middleware for HPC and Deep...
11mPer-iteration PyTorch Visibility...
25mMaking the Most Out of Your Hardware Accelerators in a Kubernetes Cluster
30mUnderstanding GPUs
01h 16mOptimizing workloads with Transformers and Diffusers
22mCentimani: Enabling Fast AI Accelerator Selection for DNN Training with a Novel...
44mArchitecture Effects When Interacting with GPU Technologies