Cost-Effective, Dense, and Performant Prometheus Storage via QLC
summary
An example deployment scrapes 8,000 targets at 6–10s intervals and uses two 15TB QLC NVMe drives for WAL and long-term storage, reports 0.09 drive writes per day with a firmware-predicted 52 years lifetime and yields ~70 days retention for that workload. Uses Prometheus/node_exporter, BlueStore optimal IO-size alignment and an endurance-profiler script; HDD-induced compaction/GC artifacts disappear and month‑scale queries return quickly.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker background
- 1:51Prometheus metrics and ecosystem
- 3:38Storage challenges with Prometheus
- 7:36Hard drives vs SSDs for Prometheus
- 16:11TLC SSDs overview and limitations
- 17:56QLC SSDs for cost-effective storage
- 21:51QLC SSD endurance and real-world data
- 24:59QLC SSD performance characteristics
- 31:35Newer QLC SSD generations and trade-offs
- 36:19QLC SSDs in Ceph and RGW workloads
- 39:03Q&A and closing remarks
related talks
06mLightning Talk
28mCortex 101: Horizontally Scalable Long Term Storage for Prometheus
39mBuild a High Performance Remote Storage for Prometheus with Unlimited Time Series
14mOperational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field...
37mIntro: Cortex
30mDistributing PromQL for Fast and Efficient Kubernetes Fleet Monitor...
05mMigrating Prometheus data between different storage systems
46mPrometheus as a Time Series Database