Scaling k8s Deployments: Achieving Elite DORA Metrics
summary
Adobe scales deployments to 15,000 namespaces and 200,000 services to achieve a 9x deployment frequency increase, cutting average commit-to-production waits to about three days and restoring services in roughly an hour. Releases use Helm charts and Flux controllers with a custom orchestration controller creating a HelmRelease per namespace, a standardized validation metric and binary quality gate, and a version orchestrator that automates phase rollouts (percentiles, randomized batches) to avoid API/resource overload. Observability uses Prometheus, Cortex and Grafana, and SlackOps provides transparent status, freezes and emergency patches.
AI-written summary · about
chapters (17)
- 0:00Scaling Kubernetes deployments
- 1:06Adobe Experience Manager
- 3:35Scale challenges with 15k namespaces
- 7:06Kubernetes deployment pipeline
- 10:19Zero downtime rollouts
- 11:42Rollout validation metrics
- 13:09Observability for 200k services
- 14:36Quality Gates for deployments
- 17:01SLO thresholds for rollouts
- 19:10Automating deployment decisions
- 20:33Phase rollouts to production
- 21:59Canary rollout risks
- 24:10Engineer feedback needs
- 25:14Slack for rollout transparency
- 27:24Lessons learned
- 30:13Open source technologies
- 31:08Q&A on Quality Gates
related talks
38mScaling Kubernetes Deployments Across +15'000 Namespaces
24mReady to Go - Measuring Release Fitness at Scale
36mOptimizing Resource Usage in Kubernetes
38mScaling Organizations Using Kubernetes
22m10 Kubernetes is hard!
28mScaling in Kubernetes Safely on On-Prem KaaS Across 1,300+ Clusters and 40,000+ Nodes
29mOrchestrating 10,000+ Kubernetes deployments with 1 pipeline
45m200 Teams, 1 Goal: DevOps Maturity at Scale