Architecting Growth: Scaling Tactics for Prometheus Metrics...
summary
An end-to-end example shows an IoT farm growing from 100k → 1.5M → 10.5M time series, forcing Prometheus from 1GB to ~70GB RAM before hitting OOM and single‑point‑failure problems. Solutions include simple vertical sizing, functional sharding or hashmod sharding (or both) via Prometheus Operator shard specs, running Prometheus as node agents (work in progress), and centralizing queries and rule eval with Thanos sidecars and querier/ruler; autoscaling shards via HPAs or KEDA is demonstrated working.
AI-written summary · about
chapters (6)
related talks
39mHow To Take Prometheus Planet Scale: Massively Large Scale Metrics Installations
27mTaking Control of Metrics Growth and Cardinality: Tips for Maximizing Your Observability
31mHandling Billions of Metrics with Prometheus and Thanos
35mDoing Things Prometheus Can’t Do with Prometheus
25mUnbreakable Prometheus: Leveraging Limits for Consistent Monitoring
36mDoing Things Prometheus Can’t Do with Prometheus
30mIntro: Prometheus
26mPrometheus That Scales