Unwiring High Cardinality
summary
Shows why a single payment timing metric easily multiplies into 100K combinations and how that scales to millions of active time series, causing queries that scan 15 million samples per alert or 45 billion for month-long SLO checks. Explains two practical controls: change labels at ingestion or in storage, plus sharding, rate‑control and streaming aggregation (precomputed indexes/materialized views) to cut scanned samples (example: from 15M to ~150). Presents a working outcome: a fully managed OpenTelemetry/Prometheus‑compatible time‑series warehouse (white paper available) that implements these techniques.
AI-written summary · about
chapters (12)
- 0:00Introduction to high cardinality
- 1:48What is a metric
- 5:45Metric cardinality explained
- 12:34Why cardinality matters
- 17:07Growing dimensions in metrics
- 23:08Where to handle cardinality
- 29:31Managing high cardinality workflow
- 34:04Flow and locality control
- 39:07Solving cardinality abuse
- 42:48Q&A on indexes and discovery
- 48:25Metrics vs event views
- 50:58Third-party vs self-hosted systems
related talks
13mLightning Talk: Taming Metric Cardinality: Practical Cost Reduction Wit...
24mScaling Telemetry To Terabytes: Applying Google's Principles To OpenTelemetry...
20mThe Sins of High Cardinality
23mDesigning for High-cardinality Metrics
27mTaking Control of Metrics Growth and Cardinality: Tips for Maximizing Your Observability
11mAggregating, Alerting and Graphing on Millions of Prometheus T...
11mLightning Talk: Beyond Billions: Operating Thanos, Prometheus & OpenTelemetry...
20mWhat if Cardinality was free?