How to Test Artificial Intelligence and Machine Learning Solutions?
summary
Demonstrates testing a Titanic ML classifier built with Python and pandas: trains logistic regression, improves ROC AUC from 0.77 to 0.85 via feature engineering, and moves the model to production as a Flask microservice. Includes unit tests for feature extractors, integration checks comparing notebook predictions to API responses, and runtime logging/monitoring to detect model drift and reproducibility issues.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 3:22Testing matters for ML
- 6:31Data exploration
- 11:30Train-test split
- 14:17Training first model
- 21:00Feature engineering
- 28:50Model deployment
- 32:05Multi-level model testing
- 35:39Production monitoring
- 42:22Tester's role in ML
- 45:28Overfitting problem
- 48:00Fighting model regression
- 50:07Customer model acceptance
related talks
01h 23mMachine learning in Action
27mUsing Machine Learning to solve a classification problem with scikit-learn
50mBuilding Intelligent Mobile Apps using Machine Learning
27mPyTesting the Limits of Machine Learning
47mML: Explainable Artificial Intelligence explained
53mUnleashing the Black Box: Testing Machine Learning Systems
45mHow to Test Artificial Intelligence and Machine Learning Solutions?
50mBuilding Intelligent Mobile Apps using Machine Learning