Uber's Synthetic Control

on YouTube ↗23m

" summary

When a citywide change prevents A/B testing the team builds a counterfactual by weighting other cities and past data using Uber's Python package and a synthetic control model implemented on Spark infrastructure. The claim is that city-level causal effects are recoverable at scale for clear signals like changes in cash trips per driver, but small marginal effects, simultaneous local shocks, and lack of preregistration limit sensitivity and invite specification checks.

AI-written summary · about

" chapters (17)

1. 0:00Welcome!
2. 0:36What to do when A/B testing is not possible
3. 2:00Motivating example: Cash trips at Uber
4. 3:47Experiment: Tell drivers if a trip is cash
5. 5:07Why an A/B Test doesn't work here: Spillover effects
6. 6:53What to do instead of A/B: Look at before and after
7. 7:40What to do instead of A/B: Switchbacks
8. 8:46What to do instead of A/B: Difference-in-Differences
9. 10:21A/B Alternative: Synthetic Control
10. 11:47How do we form our synthetics?
11. 13:42How do we estimate treatment effect?
12. 14:57Synthetic control performance
13. 15:32Synthetic control implementation at Uber scale
14. 16:25Other use cases for Synthetic Control at Uber: Uber POOL
15. 18:17Other use cases for Synthetic Control at Uber: Surge pricing
16. 19:04Limitations of Synthetic Control
17. 22:06Key takeaways

" related talks

• 32mCausal inference and scenario generation within Just Eat Takeaway.comMax Knobbout · PyData Eindhoven 2022
• 11mCausal Inference For Socio-Economic & Engineering SystemsAnish Agarwal · Meet the Fellows Welcome Event
• 26mUber’s Intelligent Insights AssistantFranziska Bell · MLconf SF 2019
• 34mData Drives BusinessThorsten Dirks, Carlo Ratti, Antoine Blondeau, Mark Bergen · DLD Munich 2016
• 32mWhy start using uplift models for more efficient marketing campaignsJing Pan · Data Con LA 2020
• 34mBalancing Weights For Causal Effects With Panel Data: Some Recent Extensions To The Synthetic...Avi Feller · Learning from Interventions 2022
• 01h 00mThe Synthetic Data Advantage: How It Helps You to Get Both AI Privacy & ExplainabilityA. Ebert · DSC Europe 23
• 13mPyTorch at UberSidney Zhang · PyTorch Developer Conference 2019