Understand ML model predictions & biases with Amazon SageMaker Clarify
summary
Amazon SageMaker Clarify detects bias at three points—during data prep, after training, and continuously in production—by running a first-party processing container or Model Monitor jobs that output a JSON report with pre- and post-training bias metrics and local and global SHAP explanations. It integrates with Studio, Data Wrangler, Pipelines and returns feature attributions (SHAP), bias tables, and drift charts; Bundesliga’s xGoals model uses 16 factors, Prudential reports faster runs and plans to bake Clarify into model governance.
AI-written summary · about
chapters (12)
- 0:00Introduction to SageMaker Clarify
- 1:51Understanding bias and explainability
- 5:37SageMaker Clarify overview
- 7:46ML lifecycle integration
- 8:07Prudential’s use case
- 10:22Importance of explainability
- 15:16Demo in SageMaker Studio
- 17:44Running bias reports
- 19:15Feature importance with SHAP
- 20:48Model Monitor integration
- 21:12Visualizing metrics over time
- 23:30Summary and key takeaways
related talks
33mBuild trustworthy ML detection and monitor bias
18mUnderstand your ML models better with greater visibility and transparency
44mImprove explainability of ML models to meet regulatory requirements
41mMachine Learning Model Fairness and Interpretability with AWS
55mEthical Machine Learning
37mStreamlining Machine Learning Operations
33mBuild, train, and deploy ML models using Amazon SageMaker
31mBuild, train, and deploy ML models using Amazon SageMaker