Speeding Up Terraform Caching with OverlayFS
summary
Claim: switching Terraform provider caching to OverlayFS plus a central readonly store and per-process upper layers makes concurrent inits safe and cuts plan time by roughly 90 percent. Implementation mounts a shared lower cache and creates writable uppers per Terraform process so downloads happen only once and missing providers are lazily fetched and persisted after planning. At ~120,000 resources this moves large-state plans from hours to minutes and smaller plans from minutes to seconds, restoring the ability to plan the whole infrastructure under an hour.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:19Terraform workflow bottlenecks
- 3:45Network requests in Terraform
- 5:50Optimizing modules and state
- 9:00Provider caching with OverlayFS
- 10:24OverlayFS for concurrent caching
- 12:48Performance impact and results
- 13:51Benefits and future integration
related talks
25mScaling Terraform at
29mBreaking Down the TofuLith. How We Reduced Lead Times for Multi-Tenant Infra Changes...
39mTerraform Practices to Enable Infrastructure Scaling
15mTerraform Practices: The Good, the Bad, and the Ugly
06mIGNITE: Terraform Practice to Enable Infrastructure Scaling
31mTerraform Practices to Enable Infrastructure Scaling
30mTerraform Practices to Enable Infrastructure Scaling
22mEnabling Infrastructure as Code