Secrets at scale: GitHub’s 9-year evolution with HashiCorp Vault
summary
GitHub migrates from Vault OSS to Enterprise to handle growth from 25M to ~300M daily requests, 55,000 clients, 68,000 secret keys and 8,000 KV stores, using read-only replicas and anycast routing to cut internal latency from ~15 ms to <1 ms. Backups shrink from 21 GB/4 hours to ~5 GB/<10 minutes and restore reliably; NVMe instances and Raft tuning improve IO and unseal times. Pain points include a low unseal timeout and ~1M stale certificates that bloat Raft snapshots; Sentinel, Autopilot and HSM enable governance and compliance.
AI-written summary · about
chapters (16)
- 0:00Introduction and overview
- 1:06GitHub’s Vault at massive scale
- 3:17Early days and custom solution
- 4:43Breaking point and limitations
- 5:47Why upgrade to enterprise
- 7:33Migration hurdles and testing
- 8:38Response time improvements
- 11:51How anycast routing works
- 14:57Business continuity with anycast
- 16:25Backup and restore speedups
- 17:30Quantified performance gains
- 18:12Lessons from September spike
- 19:39Certificate cleanup incident
- 21:47Key metrics and monitoring
- 23:35Failure tolerance and maintenance
- 23:56Future improvements and next steps
