“If you Don’t Monitor your Infrastructure, you Don’t Own it!” Regain Control Thanks to Prometheus
summary
Prometheus monitors a French FedEx–scale CQRS stack (Kafka, Spark, Cassandra, Elasticsearch, Scala APIs) across ~90 servers, processing ~2M packages/day and ~20M events/day with ~200 processes. Setup uses node_exporter, JMX and Elasticsearch exporters plus homegrown exporters, systemd collector, Alertmanager wired to Slack and Jenkins to trigger remediation. Prometheus scrapes ~250 endpoints every 15s (≈15,000 metrics, 15 gigabytes a day); lessons include the scrape-interval tradeoff (5s spikes CPU, 15s cuts load) and successful automated restarts via self-healing via AlertManager→Jenkins.
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:05What we are dealing with
- 2:10CQRS Architecture
- 3:30Monitoring system metrics
- 3:49Monitoring middleware metrics
- 4:22Monitoring business metrics
- 4:44Full stack
- 5:16Data volume
- 5:42Enabling metrics: node_exporter
- 7:30Enabling metrics: Scala app
- 8:20Self Healing using AlertManager Jenkins
- 9:08Our favorite Prometheus queries
- 10:01Demo time
- 13:53How do ops use it?
- 17:19Tip: Never stop tweaking
- 18:44Tip: Mind your NTP
- 19:36Trick: Scraping interval tradeoff low scraping interval
- 21:46Golden tip
- 22:28What's next?
- 23:58Take-away
related talks
30mPrometheus Is Good for Your Small Startup
10mPrometheus for Monitoring Metrics - Database Monitoring Tools
40mThe Prometheus monitoring system
32mDemocratizing Application Monitoring with Prometheus
27mPrometheus: The Unsung Heroes
08mPrometheus in 5 minutes
34mMonitor My Socks: Using Prometheus in a Polyglot Open Source Microservices
32mAdapting Brownfield Deployments to Cloud-Native Technology