Verification: Truth in Statistics
summary
Forecast verification compares forecasts to truth and shows concrete fixes: a consistent half a degree bias explains an easy RMSE improvement, an S1 skill score shows a steady downward trend in global performance, and event checks reveal 40% of lightning strikes outside warnings and large amounts of very heavy rainfall occurring outside thunderstorm warnings. A prototype web-based tool in Python with IPython notebooks, shareable code and data runs locally or on a server to reproduce metrics and inspect raw files, enabling rapid iteration though not yet production hardened.
AI-written summary · about
chapters (22)
- 0:00Intro
- 1:46Im not a statistician
- 2:09Talk structure
- 3:00Background
- 4:17Visual Communication
- 7:10S1 Skill School
- 10:05What it can do for you
- 12:12Scatter plots
- 14:00Spatial plots
- 15:02Pvalues
- 17:54Root mean squared error
- 19:20Rsquared
- 21:25Table of Values
- 22:20Hidden Variables
- 25:04Thunderstorm Forecasting
- 25:47Lightning
- 27:40Heavy Rainfall
- 29:48Slidey Bit
- 30:36Bootstrap
- 33:30Sharing code
- 39:49Skills needed
- 40:39Tools for the future
related talks
25mVerifying and evaluating scientific results with the open source package "scores"
49mMeteorology, Climate and Python: desperately trying to forget technical details
58mTrustworthy Data Visualization
29mSolving Forecasting Problems in R and Python
33mBacktesting and error metrics for modern time series forecasting
01h 15mAB-testing by cluster
30mStats Don't Have to Be Scary: Automatic a/b Test Analysis Using Python
25mForecasting critical food violations at restaurants using open data