Building Armada – Running Batch Jobs at Massive Scale on Kubernetes
summary
G-Research describes Armada, a batch scheduler that actually runs millions of jobs per day across many clusters (tens of thousands of nodes, thousands of GPUs), using a server cluster (API/UI, Postgres, Pulsar/Redis) and remote multi-cluster executors that lease work and launch plain Kubernetes pods. It uses Dragonfly image cache, Prometheus, OPA and fair‑share queuing, hits storage and terraform scaling limits in practice, and proves Kubernetes can work for massive batch workloads.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:22What is Armada
- 3:27How we use Armada
- 4:42Core Concepts
- 6:05User Access
- 8:00Architecture
- 11:20Cluster Anatomy
- 12:23Scaling Kubernetes
- 14:08Security
- 15:11Challenges
- 17:04Successes
- 19:22Roadmap
- 21:35How to use Armada
- 22:24Questions
related talks
17mKeynote: Sailing with K8s Armada: Multi-Cluster Management with Massive Amounts of Nodes
24mKubernetes Batch Processing at Scale - A Scheduling Perspective
38mCost-Effective Scheduling of a Massive Number of Containers in Kubernetes
29mScale Your Batch / Big Data / AI Workloads Beyond the K8s...
41mBuilding & Operating a Large-scale HPC AI Cluster on Kubernetes
29mBuilding a Batch Processing Platform...
27mGen-AI at Scale: Simplifying Orchestration of Healthcare Applications Across Multi-Cluster Environme
34mMulti-cluster Orchestration System: Karmada Updates and Use Cases