Conquering Distributed Cloud Logging Without Breaking the Bank
summary
Builds a concrete pipeline on AWS using OpenTelemetry, Loki and Grafana deployed to ECS Fargate with EFS+S3, configures .NET apps via Serilog + OTLP gRPC, and measures costs showing the OSS pipeline runs roughly 3x cheaper than OpenSearch with CloudWatch subscription filters in many workloads. Compares three Loki persistence modes (WAL+EFS, EFS+S3 shipper, and TSDB v13), recommends TSDB v13 for structured logging and large volumes, and demonstrates tracing (Tempo) and metrics (Prometheus) integration.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 2:15Logging solution costs
- 7:17EFS and IAM for Loki
- 14:07Loki storage options
- 21:09Deploying Loki, OTel, Grafana
- 36:04.NET API logging
- 39:17Traces and metrics
- 45:34Q&A
related talks
40mLarge-Scale Logging Made Easy
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
31mMy Logs Cost More Than My Services
01h 02mCentralised Logging without the blood, sweat and tears
36mFinOps & Observability: technical implementation
15mThree key considerations for deploying best-of-breed observability
37mGrafana Loki: Like Prometheus, But for logs.
53mGetting started with Grafana Loki, a modern logs database