Effective Product Experimentation: Using Data-Driven Techniques for Iteration
summary
Covers concrete experiments at Atlassian: a quick-actions A/B test comparing a nested menu and a plus-only button shows both variants lift clicks (~12.9–13.1%) and a secondary-metric pick leads to shipping the nested variant; a threaded-comments prototype is built in three weeks, yields a 5% increase in core actions and 1.5-hour faster response time, then becomes a five-month full build and ships. Includes implementation patterns like A/B tests with 50/50 splits, rollout strategies (canary, gradual, targeted) and feature toggles to kill rollouts for risk control.
AI-written summary · about
chapters (13)
- 0:00Introduction to experimentation
- 1:05Data-driven decisions explained
- 2:30What is A/B testing
- 3:53Choosing experiment metrics
- 6:04Audience segmentation for tests
- 7:53Running experiments duration
- 8:35Analyzing test results
- 9:57Real-world A/B test examples
- 14:56User behavioral metrics
- 17:43Collecting behavior insights
- 18:48Feature toggles for rollouts
- 21:16Cultural shift in organizations
- 22:43Key takeaways and next steps
related talks
48mHow to use experimentation in the back-end
35mExperimentation for Engineers
23mUsing Data to Bridge Product and Engineering Teams
28mExperimenting your way to a better product
39mExperimentation and the gold standard of data champions
32mExperimentation: putting research papers into prod
27mCustomer Driven Development : What, Why & How?
52mBuilding an A/B Testing Framework that works for Developers