Hypothesis Testing Stop Criterion with “Precision Is The Goal”

on YouTube ↗30m

" summary

Claim: Bayesian HDI with ROPE does not magically fix sequential stopping bias; simulations with Bernoulli coin flips and Python notebooks show HDI+ROPE can falsely reject a true fair null about 20% of the time and stop at ~912 or worse produce a skewed decision mean (true p=0.65 appearing as 0.72). The proposed method focuses on HDI width — Precision is the goal — and a mashed-up conservative Precision is the goal algorithm (HDI width + ROPE checks) reduces false rejections and yields sample estimates that match the ground truth, at the cost of larger N.

AI-written summary · about

" chapters (10)

1. 0:00Introduction
2. 3:37Hypothesis testing basics
3. 6:26Frequentist NHST problems
4. 8:53Bayesian HDI limits
5. 13:49Bayesian simulation results
6. 18:25Precision goal approach
7. 19:26Precision goal results
8. 21:58Conservative precision improvement
9. 24:44Summary
10. 26:34Closing

" related talks

• 43mHypothesis Testing Stop Criterion with "Precision Is The Goal"Eyal Kazin · PyData London 2022
• 43mRobustness in High-Dimensional Inference TasksJelena Bradic · Robust and High-Dimensional Statistics (Simons Institute workshop, 2018)
• 28mData Driven Convergence in Statistical EstimatorsNarayana Santhanam · Information Theory, Learning and Big Data (Simons Institute, 2015)
• 32mHypothesis selection with computational constraintsMaryam Aliakbarpour · Extroverted Sublinear Algorithms 2024
• 30mBuilding fairer models for financeAndrew Weeks · PyData Global 2020
• 22mPanel DiscussionChristopher Harshaw, Liam Solus, Arnab Bhattacharyya · Learning from Interventions 2022
• 01h 02mEstimation Beyond the IID Setting (Two Vignettes)Gregory Valiant · Algorithmic Advances for Statistical Inference with Combinatorial Structure
• 35mEstimating the Value of PersonalizationEmma Brunskill · Bridging Prediction and Intervention Problems in Social Systems (Simons Institute, 2026)