Extracting Relevant Metrics with Spectral Clustering
summary
Uses spectral clustering to reduce hundreds of time-series monitoring metrics to a few representative signals by computing correlation-based similarities, forming a graph Laplacian, and selecting the number of clusters via the eigen-gap heuristic. Preprocessing rescales data and handles negative correlations; applied to blog/android/firstload metrics it finds a small number of meaningful clusters (e.g. six), revealing cross-component ties, reducing alert noise, and aiding root-cause analysis; an example implementation notebook is provided.
AI-written summary · about
chapters (9)
- 0:00Monitoring pitfalls
- 4:41User groups
- 10:03Spectral clustering basics
- 16:03Choosing clusters
- 23:03Implementing clustering
- 26:50Dynamic safety matching
- 30:21Clustering examples
- 32:12Correlation analysis
- 35:36Q&A
related talks
38mThe Impact of Regularization in Spectral Clustering
05mTurning high-velocity data into leverage for people
19mAutomatic Metric Screening for Service Diagnosis
06mPerformance is about more than time series charts
35mObservability’s Sixth Sense: Detecting Anomalies in Metrics
01h 20mClustering Data
47mAssessing the quality of a clustering
10mTurning data into leverage