Building Resilient Telemetry Pipelines: Mastering the OpenTelemetry Collector's Per...
summary
Explains how the OpenTelemetry Collector implements a durable queue by pairing the sending queue in exporter_helper (FIFO, workers dequeuing with independent retry loops) with storage extensions found in contrib, primarily the file-backed Bolt DB which requires periodic file storage extension (Bolt) compaction. Notes defaults—consumers=10, batch/request=1000—shows Redis and SQLite are available, highlights otel_collector_queue_size metric, warns about Kubernetes scale‑down risk, and stresses persistent queue is useful but not a silver bullet.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 1:48Persistent queue overview
- 6:00Core components
- 9:34Configuration tuning
- 13:32Advanced features
- 15:24Monitoring and metrics
- 16:39Frequently asked questions
- 22:06Recap and final asks
related talks
34mReliable OpenTelemetry at Scale: No Queue, No Problem
25mDebugging the Pipeline: Observability for Your Observability with OTel + Prometheus
23mControlling Data Overhead with the OpenTelemetry Collector
25mOpenTelemetry Collector Deployment Patterns
01h 03mGet a Grip on your Telemetry using the OpenTelemetry Collector
24mIngesting Observability Data: OpenTelemetry Collector or Data Prepper?
29mMastering OpenTelemetry Collector Configuration
05mTelemetry Pipelines: Your Recipe for Success