To Make an SLO, First You Must Invent the Universe: OpenTelemetry Collection From Scra...
summary
A multi‑region Kubernetes pipeline is built to do in‑region sampling with an OpenTelemetry Collector front end and a vendor dynamic sampler called Refinery, running three production regions to avoid single points of failure. Egress pressure is concrete — about 10 terabytes a day per region (≈30 TB/day global wire). Autoscaling hits memory and queueing limits, required overprovisioning; the platform drops >99% of input but delivers 100% of requested traces and runs for about $500/day.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:23Telemetry collection challenges
- 3:50Infrastructure and deployment
- 7:18Availability and reliability design
- 10:28High-volume data handling
- 13:38Diverse telemetry needs
- 15:41Configuration management
- 17:22Key takeaways
- 18:46Cost and future plans
- 19:20Q&A
related talks
34mReliable OpenTelemetry at Scale: No Queue, No Problem
26mScaling OpenTelemetry for Modern Workloads: From Mobile To Ma...
49mAdding OpenTelemetry to Production Apps Lessons Learned
27mIngesting 6.5 Tb of Telemetry Data Daily Through Open Telemetry Protocol and Collectors
31mUnlocking the Power of OpenTelemetry: Enhancing Design, Development...
31mServerless Observability: a case study of SLOs
24mUnlocking Advanced Processing Capabilities with OpenTelemetry
21mBuilding Composable OTel Pipelines: CI/CD, Testing, Team-First, and Sca...