Building a Fine-Grained and Intelligent Resource Management System on Kubernetes
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2023 · on Thursday, 2023-09-28· at 11:50· 3夹层 3M5B会议室 | 3M Room 3M5B·
summary
Katalyst is a ByteDance-built resource management layer for Kubernetes that raises utilization and performance by colocating online and offline workloads, implementing GPU sharing with 1% compute and 1 MiB memory scheduling granularity for inference, and adding topology-aware scheduling including GPU–RDMA affinity at the PCIe switch level to enable GPUDirect RDMA for distributed training. It also provides practical efficiency features like node overcommitment, instance-size recommendation, and tidal (time-varying) colocation while preserving SLOs.
AI-written summary · about
related talks
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
06m⚡ Lightning Talk: Colocating Data Workloads and Web Services on Kubernetes to Improve Resource Utilization
38mBeyond Kubernetes: Adapting To Spe...
24mImproving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI
25mExtending Kubernetes for AI | Lessons Learned From Platform Engineering
35mResource Management WG Deep Dive
33mFrom spreadsheet scheduling to Kubernetes: building an on-premise ML platform
31mManaged Kubernetes — Next Gen Academic Infrastructure?