Are You Sure about That?! Uncertainty Quantification in AI
summary
Evaluates Gaussian processes, deep ensembles, Monte Carlo dropout and quantile regression on a noisy 0–10 regression with σ=0.5 using two‑layer NNs (20 units), Adam, batch 128, 100 MC samples and hyperparameter tuning. Gaussian processes give the most honest extrapolation uncertainty and are data‑efficient but break on heteroskedastic noise; deep ensembles perform best overall and allow aleatory/epistemic splitting; dropout methods saturate with more data; quantile regression is the simplest and fastest to implement.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:40Outline
- 2:39Uncertainty Quantification
- 5:13Neural Networks
- 6:53Regression
- 11:48Motivation
- 12:14Gaussian Processes
- 19:00Dropout
- 24:27Quantile Regression
- 30:30Experiments
- 34:51Gaussian Process
- 35:39Convergence
- 37:01Heteroskedastic
- 37:35Non Gaussian Noise
- 39:25Summary
- 41:49Conclusion
- 42:48Other approaches
related talks
35mUncertainty in deep learning
34mPrediction intervals
27mModelling the extreme using Quantile Regression
36mBayesian Deep learning with 10% of the weights
31mBut what is a Gaussian process? Regression while knowing how certain you are
46mKnow What You Don't Know: Tools to Understand Uncertainty in DL
33mQuantifying uncertainty in machine learning models
30mKnowing what you don’t know matters- Uncertainty-aware model rating