Unlocking the Power of Gradient-Boosted Trees (using LightGBM)
summary
LightGBM is shown to handle missing values natively and to do categorical encoding without one‑hot preprocessing, producing a baseline on the Kaggle house‑prices task in 18 lines of code that lands near the competition mode (top ~30). It exposes a menu of objectives (including writing your own gradient/hessian) and a specialized loss that beats a two‑stage classifier+regressor and a tuned neural net on a skewed lifetime‑value problem, integrates with SHAP for per‑sample explanations, and supports early stopping, class/sample weights and GPU/distributed training.
AI-written summary · about
chapters (19)
- 0:00Start
- 0:53LightGBM: the swiss-army knife for tabular ML
- 1:28Personal introduction
- 2:42Motivating example: Kaggle house prices prediction
- 3:46But how does it perform?
- 5:41The LightGBM magic
- 7:01How does gradient-boosting works?
- 9:36What are some parameters you can tune?
- 11:17Missing values: standard answer
- 12:37Missing values: disaster averted
- 13:42Categorical encoding standard answer
- 15:33Loss functions manu
- 17:02Lifetime value (LTV) modeling example
- 18:59Custom loss functions
- 20:23Early stopping
- 20:58Sample/class weights
- 22:23Other cool features
- 23:34Interpretability: SHAP
- 25:06Conclusion and takeaways
related talks
41mBetter Than Deep Learning: Gradient Boosting Machines (GBM)
40mGradient Boosting Machines (GBM): from Zero to Hero (with R and Python code)
18mCompetitive Analysis of the Top Gradient...
38mCatBoost - the new generation of gradient boosting
32mDeep Dive into scikit-learn's HistGradientBoosting Classifier..
31mDo You Want To Build A Forest?
01h 28mMastering gradient boosting with CatBoost
01h 33mCrunching Your Data with CatBoost - the New Gradient Boosting Library