Model Performance Monitoring - A Practitioner's Perspective
summary
Claims a deployable Model Performance Management platform that continuously detects concept, data, label and feature drift, attributes degradations with a prediction drift impact score, and surfaces meaningful alerts tied to AUC/FPR and statistical tests like PSI, KL/Jensen-Shannon and KS. Connects to S3, Hadoop, Snowflake and to scikit‑learn, TensorFlow and PyTorch. Targets high‑risk use cases (credit, fraud, healthcare), diagnoses pipeline errors vs true distribution shifts, and supports retrain/ensemble remediation.
AI-written summary · about
chapters (7)
- 0:00Model monitoring importance
- 3:34Model Performance Management
- 7:04Model monitoring challenges
- 10:57Drift in ML models
- 13:02Monitoring techniques
- 16:14Fiddler’s responsible AI
- 18:00Conclusion & Q&A
related talks
19mMinimize Risks and Accelerate MLOps with Model Performance Monitoring and Explainability
36mThe ML Monitoring Flow for Models Deployed to Production
18mReliable Maintenance of Machine Learning Models
12mModel Monitoring: Detecting and Analyzing Data Issues
39mAn SRE Approach to Monitoring ML in Production
16mMLOps — Understanding and Fighting Model Degradation in Production
46mProduction Model Deployment
36mContinuous Monitoring of Machine Learning Models