Engineering scalable IaC solutions to overcome cloud limits
summary
Built an IaC pattern that ingests 3,400 objects from five systems using AWS CDK / CloudFormation, S3, Upflow and Glue and that respects hard limits like the 1 MB template limit and service quotas such as 30 flows per stack unit. The solution groups flows into 30-flow stack units, sequences those units with explicit dependencies, stores configuration in DynamoDB to avoid gaps, and runs in production for over a year without major operational disruptions.
AI-written summary · about
chapters (11)
- 0:00Cloud limits in IaC
- 3:14Cloud limits today
- 7:30Well-Architected scalability
- 8:34Data ingestion challenge
- 10:18Service quotas and throttling
- 12:24CloudFormation stack design
- 15:11Stack units bypass limits
- 17:44Scaling to production
- 20:39Handling hard limits
- 22:48Cloud Development Kit
- 25:19Key takeaways
related talks
41mScaling Infrastructure as Code
44mDeep Dive: Infrastructure as Code on AWS
28mLessons Learned from Writing Thousands of Lines of IaC
57mCloud Adoption Patterns
41mContinuous Delivery Transformation: The New Way of Working
16mFrom Complexity to Clarity
48mMoving from spaghetti infrastructure to composable cloud environments
55mHow to reuse patterns when developing infrastructure as code