ABBY: A Django app to document your A/B tests
summary
ABBY is a Django app that centralizes A/B test records for a company of about 180 people at Jimdo and is now open sourced on GitHub. It provides a RESTful API, a Bootstrap front end, Elasticsearch search, and an evaluation pipeline using pandas and statsmodels. Uploading a CSV triggers an CSV auto-evaluation experimental path that reports P value, effect size, confidence interval and power (example uplift 6% CI 1.8–10.2% power 0.994), and the team reports real productivity gains.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:16A/B testing importance
- 2:23A/B testing best practices
- 5:37Statistical power in testing
- 7:02Documentation crucial for A/B tests
- 9:30Centralized A/B test documentation
- 10:34Introducing Abby
- 14:06Technical stack behind Abby
- 15:14Demo of Abby
- 20:40Evaluating A/B test results
- 25:39Benefits from Abby
- 26:20Q&A
related talks
30mStats Don't Have to Be Scary: Automatic a/b Test Analysis Using Python
26mAnti-Patterns in A/B Testing - or How does AB Testing Scale?
28mExpAn - A Python library for advanced statistical analysis of A/B tests
01h 18mA/B Testing with SciPy
29mAvocado testing framework and how can make your testing easier
29mSwiss Army Django: Small Footprint ETL
34mContinuous Deployment for webapps based on Django
30mAutoDocish: Automated-ish Dataset Documentation