Drift Detection on Irregular Time Series with Multiple Non-Uniform Seasonal Patterns
summary
A practical pipeline decomposes irregular business time series with non-uniform seasonalities using an adapted MIST decomposition, segments recent reference/deep/recovery periods with DTW, and applies weighted statistical tests so volume and missing observations change alerting. It runs at scale over millions of series, compresses history for an online millisecond online runtime, and is already deployable so it catches stepwise drops, short recoveries, and overlapping monthly/weekly effects while reducing false positives from predictable seasonality.
AI-written summary · about
chapters (25)
- 0:00Drift detection in payment series
- 1:10Adan underperformance and drift
- 4:02Drift in irregular series
- 6:54Challenges capturing all drift types
- 7:19Four-step drift detection
- 9:51Complex seasonality in data
- 10:35Handling missing irregular series
- 11:38Volume importance in analysis
- 12:40Segmenting reference and drift periods
- 14:48Removing patterns with MSTL
- 18:20Decomposition techniques comparison
- 19:22Adapting algorithms for irregular series
- 20:05Offline to online conversion
- 21:30Segmentation with Dynamic Time Warping
- 23:39Drift significance testing
- 24:44Business alerting metrics
- 25:48Q&A: Fast detection and DTW
- 27:11Q&A: Time refinement and matching
- 27:55Q&A: Volume weights and significance
- 28:58Q&A: Reference and drop selection
- 30:47Q&A: LSTM and anomaly detection
- 31:52Q&A: Clustering millions of series
- 33:15Q&A: Expert seasonality handling
- 34:38Q&A: Multivariate series considerations
- 35:41Q&A: Reducing monitoring redundancy
related talks
20mSmart Monitoring System for Anomaly Detection on Business Trends
25mCase Study in Travel Business - Time Series Analysis with Seasonal Data
12mIt's Time for Py: Time Series Clustering and Anomaly Detection
24mMachine learning and financial time series analysis
30mIrregular time series and how to whip them
41mReliably forecasting time-series in real-time
30mDetecting drift: how to evaluate and explore data drift in machine learning systems
29mA dive into time series for the energy sector