Understanding, Generating, and Evaluating Prediction Intervals
summary
Prediction intervals are often more useful than point estimates and Conformalized Quantile Regression gives a simple, model‑agnostic way to get them by first fitting lower and upper quantile models (e.g., 10th and 90th percentiles) then expanding those bounds using a held‑out calibration split so empirical coverage matches the target, e.g., marginal 90% coverage. Implementations include int_conformal_quantile (R, random forests) and MAPIE (Python). Main caveat is the usual exchangeability / iid assumption.
AI-written summary · about
chapters (11)
- 0:00Prediction uncertainty challenges
- 1:06Prediction intervals in decision-making
- 3:13Uncertainty and stakeholder agency
- 4:40Confidence vs prediction intervals
- 6:25Outputting prediction intervals
- 7:50Evaluating prediction intervals
- 8:33Linear regression interval weaknesses
- 10:01Conformal prediction for robust intervals
- 11:49Quantile regression for flexible bounds
- 13:58Conformalized quantile regression
- 14:49Q&A
related talks
34mPrediction intervals
30mConformal Prediction: uncertainty quantification to humanise models
22mConformal Inference with Tidymodels
33mAdaptive Prediction Intervals
37mQuantifying Uncertainty in Time Series Forecasting
31mUncertainty quantification: How much can you trust your machine learning model?
36mQuantifying uncertainty in Machine Learning predictions
30mBoosting AI Reliability- Uncertainty Quantification with MAPIE