Conformal Inference with Tidymodels
summary
Code and results for using tidymodels to get prediction intervals: three concrete methods are implemented — split conformal (fit on training, compute residuals on a 500-point calibration set), cross-validated conformal (10‑fold CV with saved fits), and conformalized quantile regression using Quantile Random Forests (QRF with 2,000 trees). Simulations show roughly 90% average coverage (CV variant sometimes ~80% on small samples); intervals can be constant-width or heteroscedastic depending on method.
AI-written summary · about
chapters (12)
- 0:00Conformal inference intro
- 1:52Prediction intervals
- 3:19Quantile conformal basics
- 6:11Advantages of conformal
- 7:36Limitations and extrapolation
- 8:38Demo: Tidymodels + SVM
- 9:20Split conformal in Tidymodels
- 10:45Cross-validated conformal
- 12:51Quantile regression for intervals
- 15:21Performance validation
- 16:44Future work and classification
- 18:50Q&A
related talks
19mUnderstanding, Generating, and Evaluating Prediction Intervals
33mAdaptive Prediction Intervals
30mConformal Prediction: uncertainty quantification to humanise models
31mUncertainty quantification: How much can you trust your machine learning model?
37mQuantifying Uncertainty in Time Series Forecasting
36mQuantifying uncertainty in Machine Learning predictions
35mUncertainty Unleashed
16mIntro to Workboots: Make Prediction Intervals from Tidymodel Workflows