Lightning Talk: Optimize the Ray Schedule and Autoscaling on Kubernetes Cloud: A Hetero...
summary
Describes a concrete optimizer for Ray on Kubernetes that replaces uniform thresholds with a task-type-aware scheduler and autoscaler that chooses VM flavors by computed resource requests rather than one-size-fits-all scaling. Implements GPU type checks and PCI locality, runs experiments on a single-server dev cluster with eight GPUs across three K8s nodes, finds vertical upsizing sometimes beats adding uniform nodes, and plans to add NVLink/MIG/RDMA support and tune the Ray in‑memory object store (Plasma/Arrow) for tighter latency. heterogeneous VM autoscaling PCI locality
AI-written summary · about
related talks
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
28mRay on Public-Cloud k8s - experiments, lessons & best practices
31mRay at Scale: Apple's Approach to Elastic GPU Management
38mBeyond Kubernetes: Adapting To Spe...
26mIntroduction to Distributed ML Workloads with Ray on Kubernetes
19mScaling Frontier AI with Ray on Kubernetes
05mLightning Talk: Not One Size Fits All, How to Size Kubernetes Clusters - Jeff Sloyer, IBM
05mSponsored Keynote: The Cloud (And Your $$$) is Not Infinite: Dynamic Scheduling at Every Layer