Building Datalakes for an International Bank with HashiCorp Terraform
summary
An international bank builds its datalake platform by packaging Terraform templates and tooling into a Docker delivery image that is tested and released via AWS CodeBuild/CodePipeline, stored in ECR, and uses S3+DynamoDB state locking and KMS encryption. Templates use Jinja+JSON, bash wrappers provide a consistent CLI, and Robot Framework runs deploy/destroy and upgrade tests. One datalake deploys in 20–30 minutes and the pipeline supports 50 parallel deployments; upgrades are validated by applying the new image over the existing release.
AI-written summary · about
chapters (8)
- 0:00Introduction to the challenge
- 2:05Infrastructure as a platform
- 10:30Development process transformation
- 13:43Tooling and automation strategy
- 16:55Environment and project structure
- 23:36CI/CD pipeline workflow
- 27:49Testing and quality assurance
- 29:32Q&A and closing remarks
related talks
29mAccelerate serverless deployments using HashiCorp Terraform in AWS
52mBuilding Database Using Terraform in AWS for Maximal Isolation Usability
21mAccelerating cloud transformation with Terraform: A World Bank success story
26mBuilding the Ultimate Databricks Environment
35mAWS like Rest API in a Large Bank
42mOrchestration of Blue-Green deployment model
01h 03mCloud Devops
28mBuilding K8s on AWS & Enabling Developers to Deliver. What to Do Better - Greg Sansom & Fahad Tariq