KubeFlux: An HPC Scheduler Plugin for Kubernetes
summary
KubeFlux implements the Fluxion scheduler as a Kubernetes Scheduling Framework plugin, running Flux in a sidecar and talking gRPC to convert PodGroup requests into all-or-nothing batch scheduling. It exposes a graph-based resource model and supports exclusive allocations and placement algorithms (pack/spread). On a three-node OpenShift cluster (48 vCPUs, 192 GB per node) LAMMPS runs with 32/64 NPI workers show better timesteps/day with KubeFlux’s pack policy versus the default scheduler; larger GROMACS experiments on a 34-node cluster also favor KubeFlux.
AI-written summary · about
chapters (11)
- 0:00Introduction and background
- 0:55The state of supercomputing
- 3:00Converged computing needs
- 8:14Scheduling challenges
- 10:26Introducing the Flux framework
- 12:52Graph-based resource management
- 15:29Klux integration details
- 19:48Scheduler architecture
- 22:50Performance evaluation
- 25:15Benchmarking with GROMACS
- 27:17Conclusion and future outlook
related talks
15mLightning Talk: Fluence: Approaching a Converged Computing Environm...
36mManaging Thousands of Clusters and Their Workloads with Flux
37mComputational Fluid Dynamics (CFD) Analysis With Kubernetes, Kubeflow, And OpenFOAM
30mThousands of Virtual Kubelets: 1-to-1 Mapping a Supercomputer To Kubernetes With Supernetes
42mIntro: Flux
33mFlux Ecosystem Evolution
07mFluxCD - Gitops For All Sizes
39mEnhance Your Kubernetes Experience with GitOps & Flux