Otel Collector: The Swiss Army Knife of Observability
summary
Adobe replaces Jaeger agents with the OpenTelemetry Collector (rolled out April 2020) to centralize traces, metrics and logs, ingesting 330 million series/day, 3.6 TB of spans/day (peaks ~150k spans/sec) and 1.2–1.3 PB of logs/day. A pre-baked Java image auto-instruments apps to send OTLP to local collectors, configs live in GitOps via the OpenTelemetry Operator, and collectors run as daemonsets or sidecars. Built components include a custom auth extension, attribute and memory_limiter processors and a span-to-metrics processor for sampling, enrichment and multi-backend export.
AI-written summary · about
chapters (20)
- 0:00OpenTelemetry at Adobe
- 1:25Observability scale today
- 2:50Why adopt Collector
- 4:56Collector in pipelines
- 6:00OpenTelemetry in Java
- 8:29Custom processors
- 10:55Multiple backends
- 12:19Edge processing & sampling
- 13:23GitOps & Operator
- 15:10Security & extensions
- 17:37Key processors
- 19:25Other Collector uses
- 19:46Data quality & rate limiting
- 22:58Trace-first troubleshooting
- 24:01Future plans
- 26:07Key takeaways
- 27:55Q&A: Span data
- 29:54Q&A: Deployment trade-offs
- 31:02Q&A: Data reduction
- 33:51Q&A: Starting advice
related talks
32mOpenTelemetry feat. logs, metrics and traces
49mAdding OpenTelemetry to Production Apps Lessons Learned
51mFull-Stack Observability: One Tool to Rule Them All
37mObservability For You And Me With Opentelemetry
50mBootiful OpenTelemetry
36mObservability For You and Me with OpenTelemetry
13mManaging Observability Data at the Edge with the OpenTelemetry Collector and OTTL
35mOpenTelemetry for Busy People