Distributing PromQL for Fast and Efficient Kubernetes Fleet Monitor...
summary
Thanos implements query pushdown (feature-flagged) to pre-evaluate idempotent PromQL functions at Prometheus sidecars and a reference vertical sharding prototype that splits queries by series labels (combined with existing time sharding). On a synthetic 100k-series benchmark with a sharding factor of 3 p90 latency drops from ~9–19s to ~5–5.7s and peak memory per query falls from ~1.5GB to ~0.8GB (~50% reduction). Current support covers grouped aggregations like sum_rate, increases request parallelism, and needs attention for S3 request patterns and wider PromQL coverage.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:01Current State of Prometheus
- 5:57Speed Up Queries
- 7:32Current Limitations
- 8:24Query Execution Engine
- 10:55Horizontal Sharding
- 11:41High Cardinality Problem
- 18:12Memory Usage
- 22:39Vertical Sharding
related talks
23mConnecting Thanos to the Outer Rim via Query API
31mHandling Billions of Metrics with Prometheus and Thanos
23mPromQL Compatibility Testing - Verifying Vendor Claims for End User Benefit
10m⚡ Lightning Talk: Vote on Your Favorite--Standardize This!--10 PromQL Queries That Measure CPU & Memory Usage
23mAutomatically Making Dashboards Load 100X Faster
27mThanos: Highly Available, Pluggable, Long Term Metric Storage for Everyone!
13mLightning Talk: A Drop-in System To Accelerate Metrics Observability by 100x Us...
19mStream vs. Batch: Leveraging M3 and Thanos for Real-Time Aggregation