Pricing, Supply, A/B and Switchback Testing in FoodTech
summary
Argues that delivery fee sits at a revenue sweet spot because raising price increases per-order revenue but reduces orders; local demand elasticity is approximated by a single derivative. Shows A/B tests fail in three-sided marketplaces due to supply-side interference and recommends switchback testing that alternates pricing windows (example: 4‑hour blocks, run ≥2 weeks). Evaluates A/B with t/Welch/Mann‑Whitney or permutation tests and evaluates switchbacks as a change‑point problem using sliding or EWMA/GARCH windows and a 0.95 quantile threshold from a recent stable period, tracking delivery fee elasticity and variance shifts.
AI-written summary · about
chapters (12)
- 0:00Intro: System-Level Experimentation in Food Tech Pricing
- 0:25What Pricing Is Optimizing: Short-Term Revenue vs Long-Term LTV
- 2:07The Core Trade-Off: Delivery Fee vs Order Volume (Finding the Sweet Spot)
- 3:23Demand Elasticity: Measuring How Orders Change with Price
- 4:55Experiment Options: A/B Testing vs Switchback Testing
- 6:24Why A/B Tests Break in Marketplaces: Supply Constraints & Interference
- 8:32Switchback Experiments Explained: Alternating Pricing Over Time
- 9:31Choosing the Right Switchback Window: Noise vs Seasonality
- 11:16Evaluating A/B Tests: Classical Hypothesis Testing & Metrics
- 13:29Evaluating Switchbacks: Change-Point Detection in Time Series
- 16:05Setting Change-Point Thresholds: Baseline Windows & Quantiles
- 18:34Wrap-Up: Key Takeaways and Final Summary
related talks
34mMaximizing marketplace experimentation: switchback design for small samples
44mA B Testing: Harder than just a color change
37mA/B testing at Zalando: concepts and tools
41mHow Deliveroo improved the ranking of restaurants
15mBehind the Price Tag: Architecting for a Billion Prices
41mA/B Testing with Style
32mCausal inference and scenario generation within Just Eat Takeaway.com
37mFood is personal! Design and impact of personalized recommendations in e-grocery