Verifying and evaluating scientific results with the open source package "scores"
summary
An open source Python package called "scores" claims to verify labeled n‑dimensional data at scale and actually implements over 50 metrics with per‑metric Jupyter tutorials, 100% automated tests and code reviews, and scaling via xarray and Dask. Examples show dimension‑preserving mean absolute error maps, population weighting that changes conclusions, and a rare‑event demo where Finley’s 3.6% tornado rate gives 96.6% accuracy versus a always‑no model at 98.1%.
AI-written summary · about
chapters (14)
- 0:00scores package intro
- 1:11What is scores
- 4:41Data structures in scores
- 6:26Dimensions in metrics
- 8:13Area weighting and masking
- 10:22Rare events challenges
- 12:52Extreme events evaluation
- 14:59Sensitive receivers
- 17:28Get involved with scores
- 18:11Choosing metrics
- 20:15Time or space shifts
- 21:37Custom threshold functions
- 23:01Guide to metrics
- 24:46Q&A
related talks
01h 57mResearch Reproducibility in Embedded Learning
29mPutting the data science back into LLM evaluation
45mVerification: Truth in Statistics
01h 33mUnderstanding and diagnosing your machine-learning models
01h 38mBasic Python Packages for Science
01h 24mInpsect and try to interpret your scikit-learn machine-learning models
33mAssurance Scoring Using Machine Learning and Analytics to Reduce Risk in the Public Sector
27mThe Science and Practice of Open and Scalable LLM Evaluations