Supercharged GitHub Actions monitoring with OpenTelemetry at Elastic
summary
Elastic builds a custom OpenTelemetry Collector receiver to ingest GitHub Actions logs and pair them with traces and metrics, running on Kubernetes Autopilot with on-disk batching and tuned processors to avoid memory spikes. The pipeline handles about 2,300 repositories and 10,000 builds/day, processing roughly a billion log records and dropping ~3 million (<0.3%). Implemented dashboards, SLOs and alerting (Slack), and rule-based detection that detected leaked secrets in logs, while navigating GitHub API rate limits and 10 GB ephemeral storage limits.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 4:43What is OpenTelemetry
- 7:09CI/CD observability journey
- 12:50Monitoring Kubernetes and collector
- 19:56Debugging OpenTelemetry collector
- 23:08GitHub Actions dashboards
- 28:06Traces and alerting
- 33:54Service level objectives
- 36:42Takeaways and future work
- 39:11Q&A
related talks
30mSupercharged GitHub Actions monitoring with OpenTelemetry at Elastic
30mGoing Plaid: Striving for Speed of Light in CI pipelines
43mRewriting an Old CI/CD on GitHub Actions for 280 Contributors?
27mIngesting 6.5 Tb of Telemetry Data Daily Through Open Telemetry Protocol and Collectors
36mGitHub Actions: Open Source Workflow Automation
51mContinuous delivery to AWS with GitHub Actions
52mGraphQL Observability with Elastic and OpenTelemetry
49mAdding OpenTelemetry to Production Apps Lessons Learned