How (Not) to Spend $15000 per Month on AWS Athena
summary
A data pipeline built on AWS Athena and S3 runs queries that scan entire buckets because queries lack a WHERE clause and partitions are unused, producing bills like €15,000 per month (one query hits ~40% of the cloud bill) and occasional spikes reported as €115,000 in a month in raw dashboards. Kafka feeds and daily dumps multiply data size, cost tags and AWS Cost Explorer sit unused, budgets get zeroes, and a single developer owns the product; fixes (partitioned queries, alerting, predictive anomaly detection) reduce the spend.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 3:17Who I am and what my team does
- 5:27Total cost of ownership
- 8:17Scary $15k Athena bill
- 9:44Cost awareness failure
- 13:00Athena cost drivers
- 15:36Data volume explosion
- 17:22Single-person team risks
- 18:49Cost reduction strategies
- 19:30Own your application
- 21:41Company awareness
- 23:50Budget alerts and reporting
- 25:55Managed services risks
- 27:20Engineering as buying
- 28:46Cloud service changes
- 29:27Q&A
related talks
38mBuilding Data Pipelines on AWS
01h 02mCentralised Logging without the blood, sweat and tears
15mPLUTUS : Serverless monitoring and alerting of costs incurred on AWS
19mUltra-Budget AWS: Running Scalable Apps for Pennies
49mSpending $34.66 for saving $150,000.00 a year in AWS
36mFinOps & Observability: technical implementation
42mServerless Tales from the Trenches
31mThe AWS Billing Machine and Optimizing Cloud Costs