How Machine Learning Can Achieve Data Quality Monitoring At Scale
summary
Company demo shows Anomalo running unsupervised machine learning over a ticket sales table to detect anomalies and produce root cause analysis. A time‑series model flags the average tickets metric jumping from ~10 to 13, a specific value 40 rising from 0% to >10% of rows, and a validation rule that finds 233 failing records. The system auto‑clusters yesterday versus historical samples, surfaces the offending segment (venue state = New York) and links alerts to Slack.
AI-written summary · about
chapters (9)
- 0:00Introduction and personal journey
- 3:35Current data quality monitoring challenges
- 8:11Limitations of rule-based monitoring
- 12:30Demo overview and data observability
- 14:35Unsupervised ML for anomaly detection
- 17:02Custom checks and validation rules
- 20:51Root cause analysis and automation
- 24:43Unsupervised ML for drift detection
- 27:11Conclusion and paradigm shift
related talks
30mBetter data quality with ML & visualization
31mLearning the learner: Using machine learning to track performance
42mMachine Learning In Observability Systems: From Insights To Action
12mModel Monitoring: Detecting and Analyzing Data Issues
34mML-Bound Anomaly Detection: Experiences Beyond Commodity Observability
42mTesting Machine Learning Modules in Production
19mThe Rise of AI in Anomaly Detection
12mUnleashing the Power of AI for Performance Monitoring& Anomaly Detection