Scaling Kubernetes Deployments Across +15'000 Namespaces
summary
Daily deployments across 15,000 namespaces using a FluxCD and Helm pipeline push infrastructure Helm charts, validated by simple rollout metrics (readiness/liveness, error rate, latency) and a custom orchestrator that reads Prometheus and Thanos aggregates. Result: average engineer wait drops to about three days, ~3 commits per deployment, and service restore in ~1 hour. Canary percentiles (start at 5%) prevent cluster overload after a doubled-pod incident; SlackOps controls (freeze, status) automate operator actions.
AI-written summary · about
chapters (12)
- 0:00Introduction and context
- 3:21Scale challenges in Kubernetes
- 8:01Current deployment results and metrics
- 11:10Deployment infrastructure and controllers
- 15:22Validation metrics and health checks
- 17:50Observability with Prometheus and Thanos
- 21:44Rollout process and quality gates
- 23:32Risk management in rolling deployments
- 27:50Rollout scope and percentiles
- 28:53Engineer feedback and Slack bot
- 31:23Key takeaways and lessons learned
- 34:09Q&A session
related talks
33mScaling k8s Deployments: Achieving Elite DORA Metrics
53mDeploying to Kubernetes Thousands of Times Per/Day
39mStrangling the Legacy with Kubernetes
22m10 Kubernetes is hard!
36mCD and GitOps with flux @AdoreMe
35mTo Infinite Scale and Beyond: Operating Kubernetes Past the Steady State
22mCreating a Micro Open-Source Community with Helm
29mOrchestrating 10,000+ Kubernetes deployments with 1 pipeline