Don't Build to Last: Strategies for Designing and Productionising Experiments
summary
Runs about two experiments per month, argues for backend-driven deep link variant routing on Android to decouple releases, recommends caching assignment at login and a centralized experiment repository to default to control, shows a real experiment where almost-complete Google Pay signup code was copied into a new view and viewmodel to isolate changes, and reports the experiment failed — users added cards but engaged less over 30 days — so cleanup was a single deletion and no refactor was needed.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:06Experiment lifecycle
- 3:14Hypothesis-driven development
- 5:00Experiment code planning
- 7:51Mobile experiment implementation
- 11:01Productionizing experiments
- 13:47Real-world case study
- 16:56Key takeaways
- 17:58Q&A
related talks
39mEscaping the Build Trap
48mExperiments, the Good, the Bad, and the Beautiful
28mThe lean architect: Experimenting your way to better decisions
46mContinuous Experimentation
44mOptimize Innovation Results with Discover, Build, Measure, Learn
36mHidden engineering challenges of A/B Testing
36m"Engineering to embrace change" Or "How clumsy engineering can lead to powerful research – Part II"
55mKeynote: Experimenting in a Sociotechnical System in Software Engineering