Learning In The Presence Of Strategic Agents: Dynamics, Equilibria, And Convergence
summary
Shows that gradient play in nonconvex general-sum continuous games often fails: many Nash equilibria are locally unstable so basins of attraction are measure zero, while zero-sum games are generically stable and discretization fixes like optimism or proximal point restore convergence. Policy-gradient on linear-quadratic games produces persistent cycles rather than equilibria, observed in up to one in four randomly sampled general-sum cases. For strategic classification, when agents are no-regret learners the learner’s relative update speed selects different Stackelberg equilibria, so learning rate changes the outcome rather than just speed.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 0:42Surge Clubs
- 1:42Overview
- 3:51Convergence
- 4:41Multiagent reinforcement learning
- 5:08Linear quadratic games
- 7:10Convergence guarantees
- 8:12Strategic classification
- 8:33Strategic classification setup
- 9:00What happens if this is repeated
- 9:38Decision Makers
- 10:32Takeaway
- 11:04Questions
related talks
01h 09mLearning in Games I
01h 06mLearning in Games II
45mPolicy Gradient: Optimal Estimation, Convergence, and Generalization beyond Cumulative Rewards
31mGradient Descent-Ascent Provably Converges to Strict Local Minmax Equilibria with a Finite Timescale
45mAGT and Machine Learning
35mOn the Global Convergence and Approximation Benefits of Policy Gradient Methods
01h 00mMin-Max Optimization from a Dynamical Systems Viewpoint
04mOn gradient-based methods for finding game-theoretic equilibria