Logging in the Age of Cost-Cutting
summary
Concrete numbers and actions: Uber cuts debug ingestion to 5% retained and saves about $8M a year, then later compresses massive jobs with 169:1 compression to drop $1.8M to $10k annually, Slack trims healthcheck noise with Fluent Bit and sees ~15% savings, Shopify uses a hot/warm/cold tier to save ~60% on retention, Razorpay swaps Splunk for Grafana Loki vs Splunk $250k→$50k and reduces bills while keeping structured JSON logs for better queries.
AI-written summary · about
chapters (13)
- 0:00Introduction and Speaker Background
- 0:42The Cost of Logging in the Cloud Era
- 1:40Understanding the Causes of High Logging Costs
- 4:09Strategy 1: Implementation Sampling
- 6:16Strategy 2: Noise Filtering
- 7:47Strategy 3: Three-Tier Storage
- 9:02Strategy 4: Open Source Solutions
- 10:00Case Study: Uber's Compression Solution
- 11:00Best Practices for Structured Logging
- 12:11Metrics and Monitoring for Optimization
- 13:21Common Pitfalls and How to Avoid Them
- 14:58Cost Optimization Roadmap
- 16:10Conclusion and Additional Resources
related talks
47mCost optimization
46mWhen the CFO Comes Knocking: Our Journey to a Cost Efficient Data Platform
10mReducing infrastructure cost during development and in production
31mThe irony of cloud cost-cutting: when saving money leads to losing money
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
31mMy Logs Cost More Than My Services
23mFinOps and Cloud Cost Optimization
17mThe Observability Money Heist - a Fintech's Tale of...