Overspending on AWS and all the great things it brought us
summary
AWS Athena queries cost $15,000/month and grow until they account for 40% of monthly AWS spend because a single SQL query scanned the entire dataset in S3 instead of filtering by day. Data lineage ran from Kafka to S3 to Athena. Remediation combines technical and organizational fixes: register services with Terraform modules, create Terraform Enterprise workspaces with cost estimates in pull requests, provide Cost Explorer dashboards and anomaly detection, and run ownership and feedback workshops so teams spot bad queries before they scale.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:27E-commerce boom infrastructure
- 3:38AWS Athena serverless appeal
- 6:09Athena query costs shock
- 8:14Athena query design flaw
- 10:44Learning at Bestseller
- 13:12Overspending root causes
- 15:00Self-service infrastructure
- 17:05Terraform service ownership
- 18:54Cost awareness across teams
- 21:01Cost monitoring anomaly detection
- 23:05Cross-team communication
- 24:33Feedback to prevent overspending
- 26:20Cost as organizational metric
- 28:30Key takeaways recommendations
- 29:33Closing
related talks
30mAnd the CFO Wept: AWS Cost Control
27mMastering AWS Cost Optimisation
27mUsage based cost optimisation at scale
31mThe AWS Billing Machine and Optimizing Cloud Costs
49mAWS Cost Reduction - Experiences and Strategies
49mSpending $34.66 for saving $150,000.00 a year in AWS
51mHow to launch enterprise transformation & optimize cost
28mTurn Data Into Insights