Monitoring Production Clusters at Scale
summary
Red Hat runs Prometheus via the Prometheus Operator, scrapes kube-state-metrics and node_exporter, and routes alerts with Alertmanager while exposing a single query endpoint that joins metrics across Prometheus instances. The monitoring stack collects tens of billions of metrics from tens of thousands of clusters and enforces namespace-level access with a proxy (from label proxy) that injects namespace labels. Platform Prometheus is being slimmed to critical metrics while delegating component-specific, HA monitoring stacks to individual teams.
AI-written summary · about
chapters (5)
related talks
06mLearnings from Running Prometheus on Google Cloud Scale
29mMonitoring: Follow the Data from the Darkest Corners
18mArchitecting Growth: Scaling Tactics for Prometheus Metrics...
25mOpen source observability for private cloud: mission impossible or not?
54mThanos: Prometheus at Scale!
37mManaging Large-Scale Kubernetes Clusters Effectively and Reliably
15mJourney from Fluent Bit, Fluentd and Prometheus to Open...
35mMake IT monitoring ready for cloud-native systems