Do You Want To Build A Forest?
summary
Shows concrete claims and working code: scikit-learn adds histogram gradient boosting (binning avoids sorts, compiled with OpenMP/Cython), matches other fast libraries on the Higgs‑Boson dataset (8.8M rows, 20 features, ~30s on 12 cores). Describes parallel tree building via joblib and p-range, explains permutation importance and partial dependence APIs, demonstrates pipelines with ColumnTransformer, and runs a live benchmark that successfully uses 12 cores.
AI-written summary · about
chapters (20)
- 0:00Forests and decision trees
- 2:27Random forests add diversity
- 4:13Gradient boosting fundamentals
- 6:41Histograms speed up boosting
- 8:04Parallelizing with scikit-learn
- 9:31Histogram boosting parameters
- 10:54Parallelization in forests
- 11:35Inspecting models with permutation
- 13:40Feature engineering with ColumnTransformer
- 16:12Limitations of impurity importance
- 17:15Handling correlated features
- 19:02Partial dependence plots
- 20:25Customizing partial dependence
- 23:59Model inspection tools overview
- 24:44Demo: histogram gradient boosting
- 26:17Future of model inspection
- 27:35Comparison with other implementations
- 28:16Improvements for tree models
- 29:49Why scikit-learn uses OpenMP
- 30:14Expanding plotting APIs
related talks
32mDeep Dive into scikit-learn's HistGradientBoosting Classifier..
11mTree Ensemble Classifiers on heterogeneous platforms
39mPushing Cython to its Limits in Scikit-learn
21mKeynote: Easier Machine Learning Thoughts From Scikit-Learn
35mScikit-Learn can do THAT?!
27mMachine Learning at Scale
01h 01mParallel Python Analyzing Large Data Sets
37mUnlocking the Power of Gradient-Boosted Trees (using LightGBM)