A/B Testing in Python
summary
A runnable Jupyter notebook generates fake ad data, implements manual proportion t/z calculations and a statsmodels proportion_ztest, and shows a nonmanual route with packages; example sample size ~1,600 uses central limit theorem, alpha = 0.05 and power ~0.8. The printed result reports t_crit ≈ 1.65 and p = 0.6, so the experiment fails to reject the null. Notes cover randomization pitfalls, production deployment hints, and a Bayesian alternative via PyMC3.
AI-written summary · about
chapters (7)
- 0:00A/B testing in Python
- 2:16Statistical concepts
- 8:35A/B testing methodology
- 15:42Hypothesis testing
- 27:35Analyzing with t-tests
- 35:11Interpreting outcomes
- 37:19Bayesian approach
related talks
18mData Science Careers via A/B Testing
33mHow to A/B test with confidence
25mMake a Difference with Simple A/B Testing
32mAB Testing to Determine Product Priority
18mBuilding an AB testing analytics system with R and Shiny
27mThe reality of testing in an artificial world
38mTesting, Testing: On Experimental Drift and Data Driven Product Design
28mOptimizing your A/B testing roadmap