What if Cardinality was free?
summary
Observability vendors charging by time series drives metrics and tag pruning; the proposal makes cardinality free so every metric keeps full dimensions. A prototype ingests metrics (examples from a Netflix-like streaming service), lets users group by movie ID on movie_play_errors, click a line to run metric joins across tables, and uses PCA to surface cohorts (CDN_offline + states) that explain spikes. Result: fewer blind spots, fewer handoffs, and measurable reduction in meantime-to-understand versus today’s sampled or dropped high-cardinality data.
AI-written summary · about
chapters (12)
- 0:00Introduction to dream
- 1:46Background and career journey
- 2:49What is cardinality?
- 4:15Misaligned incentives in observability
- 6:03Real-world impact of cost constraints
- 7:47Relational database analogy for metrics
- 9:14System metric troubleshooting example
- 10:23Typical troubleshooting timeline
- 12:07AI-assisted metric analysis
- 13:51Future tooling ideas
- 15:36Measuring meantime to understand
- 16:59Q&A: Storage system requirements
related talks
14mMetrics Without Limits
20mThe Sins of High Cardinality
13mLightning Talk: Taming Metric Cardinality: Practical Cost Reduction Wit...
56mUnwiring High Cardinality
11mLightning Talk: “Metrics That Lie”: Understanding OpenTelemetry’s Cardinality Capping...
19mCutting Through Metrics Cardinality Noise with VictoriaMetrics
23mDesigning for High-cardinality Metrics
31mPrecision Over Proliferation: SRE Approach for Leaner, Smarter and Data-Driven..