Scaling Argo CD: From Symptoms To Solution
summary
Constant 32,000 reconciliations explains runaway CPU: HPA status updates trigger reconciles, so enabling debug logs for 30 minutes identifies offending resources, then setting ignore-resource-updates for status and adding a jitter (example: 30m) smooths spikes. Memory grows to ~100 GB from cluster-wide watches; applying cluster-namespace filters or RBAC cuts it (claimed 95%), or use sharding to 15 GB per controller. Also monitor repo-server queues, pprof profiles, Argo 3.0 client-side metrics and the 500-connection API limit.
AI-written summary · about
chapters (19)
- 0:00Scaling Argo CD
- 1:05Scaling problems and symptoms
- 2:08High CPU usage diagnosis
- 4:56Reducing reconciliation events
- 7:23Mitigating CPU spikes
- 9:48Why spikes harm performance
- 11:09Fixing spikes with jitter
- 12:14Orphan resource monitoring impact
- 13:18CPU and reconciliation fixes
- 14:05High memory usage investigation
- 15:52Reducing memory with sharding
- 17:35Why Argo CD feels slow
- 18:20Work queues and dependencies
- 19:24Adjusting parallelism
- 20:05Monitoring Kubernetes API
- 21:06Cluster namespace filters
- 22:14New scaling bottlenecks
- 22:57Configuring scalability
- 23:20Conclusion
related talks
10mKeynote: Pushing the Limits: Scaling ArgoCD’s Repo-Server To Handle 30...
18mUnveil the Secret Ingredients for Argo CD in the Enterprise-Scale
29mTuning Argo Rollouts for Thousands of Workloads
26mGrowing Pain: Scaling to 10K Workflows per Week
23mMastering Argo Workflows at Scale: A Practical Guide to Scalability Excellence
29mUnleash ArgoCD Observability Superpowers
26mGitOps Me Some of That! Managing Hundreds of Clusters with Argo CD
31mHundreds of Clusters Sitting in a Tree with Argo CD