Understanding and diagnosing your machine-learning models
summary
Claims concrete diagnostics not platitudes: uses scikit-learn metrics (explained variance, mean absolute error), cross_val_score, DummyClassifier and permutation tests to establish baselines, and compares ROC AUC vs average precision for imbalanced classes. Demonstrates LIME local explanations and partial dependence plots on the Boston housing data, inspects random-forest feature importances, and shows CV bagging and nested CV for hyperparameter selection. Quantifies uncertainty (100 samples → ~±10% accuracy spread), warns about nonindependence across folds and unstable feature selection.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 3:04Model performance metrics
- 23:26Cross-validation pitfalls
- 39:46Underfitting vs overfitting
- 49:42Interpreting predictions with LIME
- 1:03:43Linear models relationships
- 1:13:47Random forests importance
- 1:17:58Partial dependency plots
- 1:22:24Model comparison testing
- 1:29:04Supervised vs unsupervised trade-offs
- 1:32:28Summary
related talks
01h 24mInpsect and try to interpret your scikit-learn machine-learning models
37mCreating correct and capable classifiers
35mVisual Diagnostics for More Effective Machine Learning
15mInterpretable Machine Learning with R
34mInterpreting Machine Learning Models: Why and How!
40mMachine Learning Model Evaluation Metrics
39mVisual diagnostics for more informed machine learning
39mDiagnosing Machine Learning Models