AB Testing to Determine Product Priority
summary
A/B testing is used to pick what to build by running real experiments on the site with tools like Adobe Target, Optimizely, Adobe Analytics and Quantum Metric, tracking lift and revenue via a CXL-style calculator and Airtable triage. Tests run 3+ weeks, aim for minimum 1,000 conversions/month, and accept a roughly 86% failure rate to learn cheaply. B2B can be pre-sold with split emails. Results translate into ROI (example: $300k incremental with $50k build cost) and are validated post‑release.
AI-written summary · about
chapters (10)
- 0:00Introduction and objectives
- 1:30Speaker introductions
- 2:35Basics of A/B testing
- 8:53A/B testing and design thinking
- 15:12Criteria for A/B testing
- 18:41Prioritizing with A/B test results
- 21:51Determining sample size
- 25:20Tools for tracking tests
- 26:45Convincing executives to test
- 30:55Resources for learning
related talks
09mDemystifying the Dreaded A/B Test
28mOptimizing your A/B testing roadmap
45mHow we killed 80% of features and increased outcomes of a/b testing by 100%
24mDoes A/B Testing Work?
28mBeyond A/B: Test automation approaches for web variants from A to Z
38mTesting, Testing: On Experimental Drift and Data Driven Product Design
39mA/B testing for 250 million people
33mHow to A/B test with confidence