Rebel: Combining Deep Reinforcement Learning and Search for Imperfect-Information Games
summary
ReBeL combines counterfactual regret minimization with deep value nets and search over public belief state rooted subgames, proving convergence to a Nash (minimax) equilibrium in two‑player zero‑sum games. It trains by self‑play, solves subgames with CFR while using a value network at leaves, and achieves superhuman results in two‑player no‑limit Texas Hold’em (≈165±69 chips per hand versus a top pro) while using far less domain engineering than prior poker bots. Remaining challenges: huge hidden‑information spaces and learning dynamics.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 3:38Alpha Zero Overview
- 6:04What is Search
- 7:33Alpha Zero
- 15:31Regret minimization
- 18:41Regret matching theorem
- 19:32Counter factual regret minimization
- 22:07CFRD
- 25:43Optimal Policy
- 27:48Decomposition
- 31:22Value Network
- 32:10Algorithm
- 37:20Comparison
- 41:18More Information
related talks
40mGeneral Game-Theoretic Multiagent Reinforcement Learning
44mThe State of Techniques for Solving Large Imperfect-Information Games
46mThe Building Blocks of Superhuman Poker AI
56mA Tutorial on Reinforcement Learning II
31mDesigning game AI with reinforcement learning
56mNo-Regret Learning in Extensive-Form Games
01h 03mIEEE CoG 2019 - Industry Day - David Silver (DeepMind)
15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust Control