Search Relevance: A/B testing to Reinforcement Learning
summary
Compares A/B testing with reinforcement learning for e‑commerce search relevance and claims multi‑armed bandits give faster convergence than A/B when quick wins matter. Implements an epsilon‑greedy bandit in Python (notebook on GitHub), using pandas, seaborn and Jupyter/Conda, simulating four ranking algorithms over SRPs of 500 items. Simulation shows the chosen algorithm often dominates within ~10–20 SRP displays and varying epsilon trades exploration for speed. Prototype works in simulation and is presented as a deployable feedback loop alternative to long A/B tests.
AI-written summary · about
chapters (27)
- 0:00Search relevance intro
- 1:31Why relevance matters
- 2:33Evolution of retrieval
- 4:17Defining relevance
- 6:27Multi-armed bandit problems
- 7:53Bandits solve exploration-exploitation
- 9:42A/B tests vs bandits
- 13:54Bandit algorithm types
- 16:23Epsilon greedy deep dive
- 18:08Bandits for search relevance
- 22:30Math behind bandits
- 26:10Live demo setup
- 27:16Mock search data
- 33:04Simulating result pages
- 34:56Bandit probability init
- 35:59Running experiments
- 37:00Building agent classes
- 40:30Explore vs exploit in code
- 41:54Algorithm selection sim
- 44:46Testing epsilon values
- 45:57Zero exploration results
- 46:4110% exploration results
- 47:4640% exploration results
- 49:55High exploration results
- 51:00Practical considerations and limitations
- 52:11When to use bandits
- 54:24Final recommendations
related talks
32mMulti-Armed Bandits in Recommendations
21mLeveraging Multi-Armed Bandit Algorithms for Dynamic Decision Making
17mByron Galbraith, Chief Data Scientist, Talla, NYC 2017
15mAI-Driven Application Optimization
41mKeynote: Personalization, Bandits, and Causal inference
30mPractical Multi Armed Bandits
33mTwo Years of Bayesian Bandits for E Commerce
22mPersonalizing Explainable Recommendations with Multi-objective Contextual Bandits