How to Handle Fair Scheduling in a Private Academic K8s infrastructure
Talkat Kubernetes Batch + HPC Day Europe 2022 · on Tuesday, 2022-05-17· at 15:25· Room 4F | Event Center·
summary
CERIT-SC runs a single multi-tenant Kubernetes cluster with 2,500 CPU cores, 130 users, ~1,000 pods/day alongside an HPC system of 30,000 CPU cores, 3,000 users, 20,000 jobs/day and faces bursts of long-lived interactive pods versus queueable HPC jobs. Problems include chronic over-requesting of resources, inability to reserve nodes or estimate pod wait time, and unsafe eviction of stateful jobs; manual priority tweaks don’t scale. Exploration focuses on non-restart resource changes and checkpoint/restore (CRIU) work in progress but fair-share reservations and lifecycle reclaiming remain unsolved.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 0:25Czech research infrastructure
- 1:23K8s infrastructure
- 2:23Challenges
- 2:57Resource Waste
- 3:35Other Problems
- 4:38Scheduling Behavior
- 6:56Outro
related talks
29mScale Your Batch / Big Data / AI Workloads Beyond the K8s...
36mCapacity Scheduling for Elastic Resource Sharing in Kubernetes
28mMultitenancy and Fairness at Scale with Kueue: A Case Study
19mKeynote: Building Bridges: Cloud Native and High Performance Computing
25mScale your Batch / Big Data / AI Workloads Beyond the Kubernetes Scheduler
29mTowards Sustainable Multi-Cluster Management in Real-World Academic Infrastructure
23mImplementation Challenges: From HPC to Containers in the Academy
10mFair Scheduling for Deep Learning Workloads in Kubernetes