ML training acceleration with heterogeneous resources in Bytedance
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Friday, 2021-12-10· at 14:05· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Describes ByteDance’s system-level work to speed up large-scale distributed deep-learning training on heterogeneous CPU/GPU clusters. Covers multiple GPU-sharing mechanisms to increase GPU utilization, NUMA-aware resource allocation (CPU, memory, GPU, NIC) to improve training performance, and integrating RDMA CNI with Intel SR-IOV for high-throughput networking.
AI-written summary · about
chapters (6)
- 0:00Intro
- 1:41GPU Offline Training (Network)
- 4:23GPU Offline Training (Scheduling).
- 5:24GPU Online Serving
- 8:21GPU Unified Scheduling
- 13:49Future Work
related talks
24mImproving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
14mEfficient Training of Video Generation Foundation Model at ByteDance
34mDeep Customized Kubernetes for Machine Learning in Tencent
26mDeepSpeed: Efficient Training Scalability for Deep Learning
03mDBTB INT Subbu Rama r
12mBehemoth: A Flash-centric Training Accelerator for Extreme-scale DNNs
26mEfficiently serving LLMs at scale.