Building an A/B Testing Framework that works for Developers
summary
A/B testing at scale is practical but costly: a deployed framework integrates with Google Analytics, qualifies users by device and custom rules, and handles distribution across brands; the app has 400k orders/day and 15M active users with 60% mobile so experiments matter. Teams build a debug settings screen, automation suites and feature toggles (or a kill switch), run long experiments — one navigation test lasts eight months — ship haptic feedback and carousels, iterate until some variants win and others get cleaned up.
AI-written summary · about
chapters (16)
- 0:00Start
- 1:20ABOUT ME
- 3:33A/B testing is
- 6:10DEAD CODE
- 10:27COACH BUBBLES First losing experiment
- 11:57SCHEDULE ORDER ENTRY POINT
- 13:10HAPTIC FEEDBACKS
- 14:05Flow example
- 14:40Multiple iterations examples
- 19:32A/B EXPERIMENTS Winning is not guaranteed
- 19:51KEYS FOR A/B TESTING SUCCESS
- 22:36LESSON #2
- 26:06LESSON 83
- 28:51LESSON #5
- 30:54LESSON #7
- 34:01TAKE AWAYS
related talks
01h 15mExperimentation at scale: Fuel your application’s growth through A/B testing
28mOptimizing your A/B testing roadmap
39mDIY A/B Testing Architecture
34mA/B Testing You'll Like
46mAB Test Bloody Everything
36mA/B Testing in Production
36mA/B Testing in Production
05mUsing A B testing to rebuild the UI used by 1 5 million users a day