Smart Monitoring System for Anomaly Detection on Business Trends
summary
Describes Alibaba’s smart monitoring system for real-time anomaly detection on hundreds of business KPIs using seasonal time-series forecasting and human feedback. It explains a preprocessing pipeline and use of STL (Seasonal-Trend decomposition using Loess) for prediction to balance computation, accuracy and robustness, and a closed-loop method that sets segmental thresholds initially and continuously updates them from manual labels to improve detection and tolerate noisy/contradictory feedback.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:37About me
- 1:14Introduction to Alibaba Group
- 2:23About business trends monitoring in Alibaba
- 3:39Features of businesses trends
- 5:32Challenges of anomaly detection on business trends
- 6:40Summary of anomaly detection approaches
- 8:19Our choice
- 9:28How to get a good "prediction"
- 9:58Using STL directly on original data...
- 11:02Customized data preprocessing
- 12:19A better "prediction" is born
- 12:53Anomaly detection based on predicted curve
- 14:23How to determine the N'S
- 15:41Manual feedback loop
- 17:09Evaluation
- 18:44Future work
related talks
20mIntroduction to Alibaba Monitoring System
30mBetter data quality with ML & visualization
35mObservability’s Sixth Sense: Detecting Anomalies in Metrics
30mMassive Scale Anomaly Detection Framework
28mHow Machine Learning Can Achieve Data Quality Monitoring At Scale
19mThe Rise of AI in Anomaly Detection
36mContinuous Monitoring of Machine Learning Models
26mProactive Failure Prediction Through Anomaly Detection