Deep Dive: CRI- RM Based CPU and NUMA Affinity to Achieve AI Task Accele...
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2021 · on Friday, 2021-12-10· at 11:20· Kubecon + CloudNativeCon 演讲厅, 虚拟会议 (Virtual)·
summary
Describes using the open-source CRI-RM component (developed with Intel) to perform in-node resource allocation in Kubernetes so CPU binding and NUMA affinity can accelerate CPU-based AI training. Explains limitations of the built-in Kubelet CPU and Topology Manager (works only for Guaranteed-QoS pods and is hard to customize) and presents a CRI-RM–based approach that exposes host topology, works on older Kubernetes versions, supports pods without memory limits, and claims over 50% AI task performance improvement.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:44Content
- 1:29CRI-RM architecture
- 5:37Noisy neighbors
- 7:31Latency critical workloaus
- 8:45CPU clock speed throtung
- 11:14Available resource and control
- 14:40Topology aware policy
- 15:25Static-pools policy
- 16:14CRI-RM node agent
- 16:50CRI-RM webhook
- 17:39Topology-aware resource alignment
- 18:21Some problems In Al training cluster (Kubernetes* + Docker*)
- 21:29Run Al training tasks on the CPU
- 23:14CPU management in Kubernetes
- 24:59Kubernetes* integrated CRI-RM
- 26:51Test environment
- 27:37Test casel: restnet50+imagenet
- 28:25Test case2: CNN+minst
- 28:56Conclusion
related talks
34mUnlocking Performance: Topology-Aware CPU Scheduling W...
23mThis is The Way: A Crash Course on the Intricacies of Managing CPUs...
36mMitigating Noisy Neighbours: Advanced Container Resource Management
18mLet There Be Topology-Awareness in Kube-Scheduler!
20mBoosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
21mDo Not Disturb Mode: Fine-Tuning Resources for Latency Sensitive Workloads
36mOptimizing Kubernetes for Low-Latency and Real-Time Containers
24mMaximizing Workload's Performance With Smarter Runtimes