Production Data to the Model: “Are You Getting My Drift?”
summary
Defines concrete drift types and detection recipes then shows a small Python notebook that injects synthetic drift into Dow Jones price data and compares metrics. Uses two‑sample tests, KS (which fails on nonnormal data), Jensen‑Shannon/KL divergences and Earth Mover’s / M distance, observing values ~1.5–1.75 as a clear signal. Mentions tooling like evidently (and NannyML), practical rules for alerts (percent of features drifting), diagnostics (logs, integrity, cohorts) and when to retrain or recalibrate.
AI-written summary · about
chapters (11)
- 0:00Data drift introduction
- 1:07Why models fail
- 2:33Data drift types
- 5:25Why data drift happens
- 7:12Detecting drift statistically
- 10:46Choosing tests based on feature types
- 12:03Live drift detection demo
- 14:59Next steps after drift
- 18:58Drift improves robustness
- 19:21Text data drift
- 22:11Q&A
related talks
30mDetecting drift: how to evaluate and explore data drift in machine learning systems
34mTraining & Monitoring AI - Drift Detection
35mMonitoring of model drift in Finance
36mThe ML Monitoring Flow for Models Deployed to Production
42mTesting a Data Science Model
31mMonitoring machine learning models in production
30mDetect machine learning (ML) model drift in production
19mMinimize Risks and Accelerate MLOps with Model Performance Monitoring and Explainability