High-Performance MPI and Deep Learning with Introspection...
summary
Describes two X-ScaleSolutions-supported packages for high-performance MPI and deep learning on OpenPOWER with NVIDIA GPUs and RDMA interconnects (InfiniBand, RoCE). One package is the OSU MVAPICH2 MPI libraries for CPU (POWER9) and GPU-accelerated HPC; the other is tight integration of OSU MVAPICH2-GDR with the Horovod stack to deliver scalable deep learning and deep introspection for TensorFlow, PyTorch, and MXNet. Presents performance results from large systems (ORNL SUMMIT and Lassen) using thousands of GPUs.
AI-written summary · about
related talks
31mHigh-Performance MPI and Deep Learning on OpenPOWER Platform
45mHigh-Performance and Scalable Middleware for HPC and Deep...
47mDesigning High-Performance Scalable Middleware for HPC, AI, & Data Science in Exascale Systems & ...
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
42mDifferential Software Solutions on HPC and IBM Power Architecture
30mDistributed Deep Neural Network Training using MPI on Python
10mvLLM & Deepspeed Updates