Predicting Faulty Validations in Cluster Issue Detection: A ML Approach
summary
A dataset of about 400+ validation rules and git history yields numeric features—code length, entropy, time since introduction, comments, unit tests—and labels from six-week release histories; classical tabular models (logistic/linear, tree ensembles like XGBoost with monotonic constraints and a piecewise-linear + tree hybrid) rank rules by future fix likelihood with measurable AUC gains, a UI shows scores and feature effects, unit tests dominate reliability, and models train in minutes while LLM baselines take hours and underperform.
AI-written summary · about
chapters (18)
- 0:00Faulty validation prediction
- 1:27Health checks for SRES
- 3:17Support engineers' tool needs
- 5:06False positives vs nightly tests
- 6:34Validation prioritization research
- 8:45Feature extraction from code and logs
- 10:14Tabular data for ML
- 11:19Explainable ML models
- 13:51Monotonic constraints for models
- 14:56ML model interface demo
- 17:05Energy vs LLM comparison
- 18:09Key insights and future work
- 18:51Acknowledgments
- 19:14Q&A on data
- 20:48Method for other projects
- 21:31Dataset size significance
- 23:21Writing reliable validations
- 24:31ML replaces nightly checks
related talks
30mHow to Properly Test ML Models & Data
35mHow implementing ML ended up with an if-else
19mApples, Oranges, and ML Models: Model Validation vs Benchmarking
32mReliable Machine Learning
05mData Quality and Validation in ML Pipelines
50mPractical Applications of ML in Software Testing
35mA simple way to validate and monitor the performance of your ML Applications.
30mBetter data quality with ML & visualization