No A/B — result XZ, or how we built an A/B testing platform at Ozon
summary
Describes how Ozon built an internal A/B testing platform that handles business needs (hypothesis validation), statistical concerns (significance), and technical constraints (low-latency routing under high RPS). Covers design decisions and engineering trade-offs for running reliable experiments at scale in an e-commerce environment.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:15Why A/B tests
- 4:04Measuring results without tests
- 8:17Response speed elasticity
- 13:23First architecture issues
- 16:19Modern microservice architecture
- 18:27Real-time solution
- 19:30Singleton design pattern
- 20:35User distribution uniformity
- 22:00Cascading query optimization
- 24:10Caching and CDN
- 25:57Analytics data sending
- 27:04Armaggedon causes
- 28:11Stage overload problem
- 29:56Data volume reduction
- 31:28Architecture optimization
- 32:32Binary compression protocol
- 35:32Optimization results
- 36:16Fast client integration
- 39:31Company process changes
- 40:55Architecture recommendations
- 42:43Q&A
related talks
24mDoes A/B Testing Work?
34mA/B Testing You'll Like
15mA/B Testing at Scale
15mA/B Testing: Test Your Own Hypotheses & Prepare to be Wrong
27mHow to A/B test with confidence
40mReal-time A/B: how to get metrics for experiments
24mThe Scientific Method of Experimentation
19mA/B User Testing When Your CDN Cache Messes With Your Results