Mastering Argo Workflows at Scale: A Practical Guide to Scalability Excellence
summary
Case study shows Argo Workflows scaled from single‑pod to 18,000 pods per workflow by tuning controller and node resources, splitting large manifests (or offloading them to a DB), and running 66 experiments down to 15 tuned regressors; pipeline runtime drops 4h→1h. Key tooling: Prometheus + Grafana dashboard for controller, API and pod pending visibility, autoscaling nodes (spot/ARM for cost), and running a single tuned controller with priority classes rather than idle HA replicas.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 0:15Meet Tim and Alec
- 0:55About PipKit
- 1:19About Fetch Analytics
- 1:57How Argo fits in with Fetch Analytics
- 2:50Parameter Grid Search
- 4:10Workflow Life Cycle
- 5:18What could go wrong
- 5:50Kubernetes API
- 7:56Request Entity Too High
- 8:50Workflow Controller
- 10:12Server
- 11:15Monitoring
- 12:43Other metrics
- 14:21Scale horizontally
- 15:30Graina Dashboard
- 16:15Workflow Metrics
- 17:20Observability
- 17:55Demo
- 19:49Registration
- 20:11Summary
- 21:37Outro
related talks
23mArgo at Scale: Navigating Complex Multi-Dimensional Deployments Across H...
18mTailored Deployment Strategies: Argo Rollouts for Diverse Applications
27mUntangling the Threads: Understanding Argo Workflows with Workflow Tracing
22mAnalysis in Action: Real-World Metrics Driven Argo Roll...
29mUnleash ArgoCD Observability Superpowers
15mMaking Complex R Forecast App...
23mAutomated Resilience: Using Argo Events for Real-Time Incident Remediation
21mApplied GitOps with Argo CD Autopilot Using Multiple Clusters with an ApplicationSet