Monitoring of model drift in Finance
summary
A deploy-gate system at Rabbo Bank automates model-drift checks by building draft states and comparing them with Azure DevOps pipelines, orchestrated via Databricks jobs and written in Python with results in a Hive metastore visualized in PowerBI. It computes single and three-month cumulative deltas, reports percent drift with green/yellow/red thresholds, and caught a nondeterministic floating-point output change plus a later production bug that was detected within a day.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:00Model drift explained
- 6:34Monitoring requirements
- 7:38TIX model
- 9:03Measuring changes framework
- 11:53CI/CD integration
- 13:01Azure DevOps workflow
- 16:14Model changes implementation
- 18:21PowerBI results
- 20:52Real-world findings
- 23:44Key takeaways
- 25:50Q&A
related talks
36mThe ML Monitoring Flow for Models Deployed to Production
34mTraining & Monitoring AI - Drift Detection
42mTesting Machine Learning Modules in Production
12mModel Monitoring: Detecting and Analyzing Data Issues
26mProduction Data to the Model: “Are You Getting My Drift?”
45mExplainable Monitoring for Successful AI Deployments
19mModel Performance Monitoring - A Practitioner's Perspective
30mDetecting drift: how to evaluate and explore data drift in machine learning systems