Improve explainability of ML models to meet regulatory requirements

on YouTube ↗44m

" summary

Explains how Amazon SageMaker Clarify runs explainability and bias checks across the ML lifecycle, producing S3/PDF reports, CloudWatch metrics and alerts, and eight pre/post bias metrics, not all satisfiable simultaneously. Shows concrete demos: an XGBoost credit-risk model with SHAP-based feature attributions (checking account status dominates) and a Hugging Face transformer NLP pipeline using sentence-level SHAP baselines. Recommends model cards, data lineage, Ground Truth labeling controls, and continuous monitoring for drift.

AI-written summary · about

" chapters (25)

1. 0:00Intro
2. 2:09Explainable AI
3. 6:07Impact of datasets in machine learning (ML)
4. 8:23What are my business outcomes?
5. 10:51Where can bias happen in an ML workflow?
6. 13:12Ethical AI
7. 16:49Model evaluation - Underfitting model
8. 18:42Trade-off between model accuracy and interpretability
9. 20:51Shapley values
10. 21:44Shapley Additive exPlanation (SHAP)
11. 22:46Explaining model predictions: SHAP algorithm Different solutions for runtime and algorithmic challenges
12. 24:26Model cards
13. 26:37Definition of a model in the financial industry
14. 28:02Stages in a model lifecycle
15. 28:56What are the regulatory requirements?
16. 30:19Operational challenges of managing the ML lifecycle
17. 30:50Challenges of ML model management - Lack of information on the ownership and traceability of the models
18. 32:11SageMaker Clarify across the ML lifecycle
19. 33:00SageMaker Clarify APIs
20. 33:32Model explainability monitoring using SageMaker Clarify
21. 34:49Model bias monitoring using SageMaker Clarify
22. 35:45Use cases
23. 40:55Summary
24. 43:06Additional reading
25. 43:33References

" related talks

• 18mUnderstand your ML models better with greater visibility and transparencyPinar Yilmaz · AWS Machine Learning Summit 2021
• 24mUnderstand ML model predictions & biases with Amazon SageMaker ClarifyMichael Sun, Pinar Yilmaz · AWS re:Invent 2020
• 41mMachine Learning Model Fairness and Interpretability with AWSAdewale Akinfaderin · DevReach 2022
• 33mBuild trustworthy ML detection and monitor biasAaron Sengstacken · AWS Summit Online – Americas 2021
• 27mInterpretability and explainability in machine learningVikrant Kahlir · AWS re:Invent 2020
• 39mGuide towards algorithm explainability in machine learningAlejandro Saucedo · PyData London 2019
• 37mStreamlining Machine Learning OperationsKapil Pendse · AWS Summit Online 2020
• 24mWhy and How to Build Explainability into your ML WorkflowKrishna Gade · TWIMLcon: AI Platforms 2019