From correlation to causation: measuring ad effectiveness at scale
summary
Describes a production pipeline that creates randomized holdouts inside a DSP to measure causal ad lift, logging bids to S3, aggregating with Spark/Scala, modeling exposure with logistic/XGBoost, and analyzing lift in R. Handles 9–10 million auctions per second across several billion cookies, runs tests from ~2 weeks to 6–9 months, and uses ghost ads (predict-exposed vs predicted-unexposed) plus KS/placebo checks to catch control contamination. Emphasizes practical limits: power, bid competitiveness, and diagnostic graphics to verify that the approach actually produces defensible incrementality. R + Spark/Scala
AI-written summary · about
chapters (18)
- 0:00Start
- 0:04Measuring Ad Effectiveness at Scale
- 1:29How Do You Know if Your Ad Actually Worked
- 1:45Media Mix Modeling
- 6:00How Do I Know Which Ads Are Working Best
- 8:58The Right Metric for Measuring Out Effectiveness
- 11:43The Challenges for a Given Ad Campaign
- 17:28Return on Adspend
- 18:30Control Group
- 21:40How Could I Set Up a Test Group in a Control Group
- 24:27Intent To Treat
- 26:53Causal Inference Methodology
- 27:13Ghost Ads
- 36:56Ks Test
- 38:46Build the Test Code in Spark
- 39:10Step 5
- 39:51Summary
- 42:10How Do We Optimize towards Lift
related talks
47mIncrementality Bidding and Attribution
29mAI and ML for Computational Advertising
36mML: Predictive analytics in Ad Engineering
28mCorrelation Sucks - Long Live Causality! Mastering Causal Inference in Data Science
21mHow to Get Ahead in Advertising
30mPredictive Analytics in Social Media and Online Display Advertising
27mLarge Scale Response Prediction to Serve Ads on LinkedIn
36mMeasuring Media Impact- Practical Geo-Lift Incrementality Testing