"Calibeating": Beating Forecasters at Their Own Game
summary
Claims an online algorithm that provably lowers another forecaster’s Brier score by at least that forecaster’s calibration loss (up to o(1) error), even against adversarial outcomes. The procedure simply predicts the empirical average of the other forecaster’s current forecast-bin from the previous day (forecast with previous-bin average). Proof uses a variance decomposition into calibration versus refinement and an online variance identity, plus a minimax / stochastic fixed point argument to handle self‑calibration and multiple experts. Extensions include deterministic continuous calibration and simultaneous calibeating of many forecasters (with practical binning caveats).
AI-written summary · about
chapters (18)
- 0:00Intro
- 12:35Perfect calibration
- 16:11Calibration plots
- 17:18Example
- 20:41Notations
- 26:01Expertise
- 27:02Calibration
- 33:16Arbitrary weather
- 34:01The procedure
- 40:41The stochastic fixed point
- 43:47The result
- 44:31The infinite regress
- 45:30Continuous calibration
- 47:26Linear programming
- 48:05Successful economic forecasting
- 49:16How to get from the end to the end
- 50:54Continuum of Forecasters
- 54:42Free Calibration