Automate Secret Rotation in Kubernetes, Then Get Out of the Way!
summary
Automated rotation uses an External Secrets Operator to sync cloud stores into Kubernetes Secrets and a reloader to trigger workload rollouts; this pipeline runs in production for ~two years but needs test secrets per store and monitoring to catch failures. A 5-hour outage occurs when store validation backoff prevents syncs; the fix is GitOps annotations to restart validation. Alternative is mutating-webhook injection with Bank-Vaults (injects env vars from Vault today, rollout-on-change work in progress).
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:48Secret rotation in Kubernetes
- 4:00Automate secret rotation
- 6:29Manual rotation challenges
- 8:15Securing Kubernetes Secrets
- 9:16External Secrets Operator
- 12:29Synchronizing with reloader
- 14:18Monitoring ESO pipelines
- 18:57Alternatives to Kubernetes Secrets
- 21:05Bank Vaults for Vault
- 27:11Live demo: External Secrets Operator
- 32:13Summary
related talks
30mLong Live Short Lived Credentials - Auto-rotating Secrets At Scale
26mThe Power of SecretOps Automating Secret Workflows
16mThe Power of SecretOps: Automating Secrets Workflows
41mOptimizing Secrets Management
24mCan You Keep a Secret? On Secret Management in Kubernetes
26mRefactoring Applications for Dynamic Secrets
29m100,000 Different Ways to Manage Secrets in GitOps
43mSecuring Secrets in the GitOps Era