Dynamic Scaling of GPUs for Container Apps with Composable Disaggregated...
summary
Built a working system that composes PCIe/CXL-attached GPUs on demand and hooks into Kubernetes autoscaling: Composable Disaggregated Infrastructure (CDI), a CDI operator (CRD) and Dynamic Device Scaling (DDS). DDS decides whether to attach GPUs or add nodes by simulating unschedulable pods using the machineSet max-GPU spec and can revert replica counts to request GPUs instead of nodes. Demo shows adding a GPU to a node and provisioning a new node (replicas 2→3) successfully.
AI-written summary · about
chapters (8)
- 0:00GPU dynamic scaling intro
- 3:10Composable disaggregated explained
- 7:52Vertical scaling challenges
- 14:56Dynamic scaling feature details
- 20:57Composable operator & resources
- 25:16Future standardization plans
- 25:38Live GPU & node scaling demo
- 33:57Q&A
related talks
41mDynamic GPU Provisioning in Kubernetes With Composable Disaggregated...
05mTowards a Sustainable World Realized by Kubernetes and Composable...
30mCoHDI: A CNCF Sandbox Project for Dynamic Hardware Composability in Kubernetes
30mRunning AI Workloads in Containers and Kubernetes
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
24mFit-to-Serve: How a New DRA Capability for Dynamic Device...
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!