Fast, intuitive feature selection via regression on Shapley values
summary
shap-select claims feature selection needs one model fit and nothing more and implements a linear or logistic regression of the target on Shapley values computed on the validation set to use coefficient sign and significance as a heuristic. experiments with XGBoost on a Kaggle credit dataset selected six features and yields similar accuracy and F1 while showing runtime much lower than Boruta; iterative collinearity pruning refits only the cheap regression and a 5% p threshold maximizes F1.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 0:20Speaker introduction
- 1:06Feature selection overview
- 3:38Filter methods deep dive
- 5:01Wrapper methods exploration
- 6:27Embedded methods understanding
- 7:51Motivation for Shapley regression
- 8:14Shapley values introduction
- 10:23Overfitting with Shapley validation
- 11:49Linear regression on Shapley values
- 13:55Feature ranking pseudocode
- 15:21Benchmarking setup
- 16:27Results comparison
- 17:53P-value thresholds impact
- 18:37Conclusion and trade-offs
- 19:41Open source release
- 20:26Q&A
related talks
30mTraditional & Novel Feature Selection Approaches
35mTaking a peek under the hood - Interpreting black box model
35mSHAPtivating Insights: unravelling blackbox AI models
34mRethinking Feature Importance-Evaluating SHAP & TreeSHAP for Tree-Based MLM
44mWhat is Your Data Worth? Equitable Data Valuation in Machine Learning
31mBeyond the Black Box: Interpreting ML Models with SHAP
17mUsing structure to select features in high dimension
26mShparkley: Scaling Shapley values with Spark