Leveraging Multi-Armed Bandit Algorithms for Dynamic Decision Making
summary
Compares multi-armed bandit algorithms by running simulations on an ABC test with conversion rates 6%, 2%, 4.5% and shows how different policies allocate traffic to the true winner over time; Thompson sampling overfits short-term trends while some algorithms are more gradual or aggressive. Describes production options: online low‑latency evaluation versus offline scheduled policy updates, recommends Monte Carlo simulations to estimate required sample size (example ~4,400 users) and adding embeddings for contextual bandits to enable personalization.
AI-written summary · about
chapters (12)
- 0:00Dynamic decision making
- 1:29Reinforcement learning basics
- 3:56Policies in reinforcement learning
- 4:17Multi-armed bandits
- 5:18Bandit use cases
- 6:45Traditional A/B testing
- 8:52Bandits vs A/B tests
- 10:17Adobe’s bandit study
- 12:48Contextual bandits
- 14:11Bandits in production
- 15:17Validating bandit results
- 18:50Key takeaways
related talks
30mPractical Multi Armed Bandits
32mMulti-Armed Bandits in Recommendations
33mTwo Years of Bayesian Bandits for E Commerce
15mAI-Driven Application Optimization
17mByron Galbraith, Chief Data Scientist, Talla, NYC 2017
41mKeynote: Personalization, Bandits, and Causal inference
56mIntroducing Decision-Making Under Uncertainty to Medical Research:...
48mAdaptivity and Confounding in Multi-armed Bandit Experiments