Multi-Agent Reinforcement Learning and Bandit Learning
2022-04-29..2022-05-04
Berkeley, United States
[Mathematics of Online Decision Making 2020, Reinforcement Learning from Batch Data and Simulation, Quantifying Uncertainty: Stochastic, Adversarial, and Beyond, Adversarial Approaches in Machine Learning 2022, Data-Driven Decision Processes Boot Camp 2022, Bridging Prediction and Intervention Problems in Social Systems (Simons Institute, 2026)]
who it's for
multi-agent reinforcement learning researchers, reinforcement learning theorists, algorithmic game theorists, bandit researchers, control engineers, reinforcement learning practitioners, graduate students
topics
decentralized V-learning algorithm; policy gradients in general-sum games; complexity of Markov equilibrium; no-regret learning in time-varying games; multi-player bandits with no collisions; mean-field linear-quadratic games; independent decentralized Q-learning; calibeating and forecast calibration; reward design for multi-agent systems; Markov persuasion and sequential information design; kernelized multiplicative weights for games; zero-shot communication in multi-agent RL
| title | author | length |
|---|---|---|
| "Calibeating": Beating Forecasters at Their Own Game | Sergiu Hart | 59m |
Claims an online algorithm that provably lowers another forecaster’s Brier score by at least that forecaster’s calibration loss (up to o(1) error), even… | ||
| V-Learning: Simple, Efficient, Decentralized Algorithm for Multiagent RL | Chi Jin | 56m |
Claim: a new algorithm V-learning lets each agent run the same single-agent procedure independently and provably finds Nash equilibria (two-player zero-sum)… | ||
| What is the Statistical Complexity of Reinforcement Learning? | Sham Kakade | 53m |
Shows that simple linear Q realizability and linear policy classes still require exponential samples in horizon or dimension, so they fail for high‑dimensional… | ||
| No-Regret Learning in Time-Varying Zero-Sum Games | Haipeng Luo | 36m |
Proposes a parameter-free algorithm that combines optimistic base learners (satisfying a dynamic RVU property) with an optimistic meta-learner over a log-T… | ||
| Policy Gradients in General-Sum Dynamic Games: When Do They Even Converge? | Eric Mazumdar | 28m |
Shows that policy gradient on general-sum linear-quadratic games fails to guarantee asymptotic convergence: Nash equilibria can be strict saddle points of the… | ||
| The Complexity of Infinite-Horizon General-Sum Stochastic Games: Turn-Based and Simultaneous Play | Vidya Muthukumar | 54m |
Shows non‑stationary Nash equilibria computable poly(S,A) and stationary Nash equilibria computable poly(A), but proves computing an approximate stationary NE… | ||
| The Complexity of Markov Equilibrium in Stochastic Games | Noah Golowich | 55m |
Proves that computing stationary Markov CCE is PPAD-hard even for two-player turn-based stochastic games with a constant discount factor and constant… | ||
| A Regret Minimization Approach to Mutli-Agent Control and RL | Elad Hazan | 42m |
Claim: multi-agent control reduces to distributed online convex optimization by linearize global cost into local gradients and letting each actuator run a… | ||
| Multi-Agent Reinforcement Learning in the High Population Regime | Tamer Başar | 53m |
Shows that for multi‑population discrete time LQG games the mean‑field equilibrium is affine and produces an approximate Nash with error O(1/√N) where N is the… | ||
| Learning Automata as Building Blocks for MARL | Ann Nowe | 30m |
Learning Automata using the Reward-in-Action update directly tune action probabilities and have a built-in exploration that, by Narendra and Wheeler’s theorem… | ||
| On Rewards in Multi-Agent Systems | Doina Precup | 32m |
Empirical claim: adding agent-to-agent rewards mitigates Tragedy of the Commons. Experiments run in the harvest domain with 10 agents, apples that only regrow… | ||
| Independent Learning in Stochastic Games | Kaiqing Zhang | 38m |
Claim: a small modification of independent Q‑learning — decentralized Q‑learning with a two-time-scale update and smooth best responses — provably converges to… | ||
| Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning | Zhuoran Yang | 38m |
Introduces a Markov persuasion process where a sender observes per-step outcomes, sends recommended actions as signals, and seeks to maximize cumulative sender… | ||
| Multi-Agent Reinforcement Learning Towards Zero-Shot Communication | Kalesha Bullard | 58m |
Paper introduces zero-shot communication and an algorithm (QED) that trains many sender/receiver pairs under a Zipfian goal prior and action energy cost, then… | ||
| Kernelized Multiplicative Weights for 0/1-Polyhedral Games:... | Gabriele Farina | 52m |
Paper proves that Optimistic Multiplicative Weights on the normal-form equivalent of an extensive-form game is simulatable with D+1 kernel evaluations per… | ||
| General Game-Theoretic Multiagent Reinforcement Learning | MARC LANCTOT, Mark Longtait | 40m |
Extends PSRO/Double Oracle to compute correlated equilibria by replacing the meta-solver with a maximum Gini-correlated equilibrium quadratic program, and… | ||
| When Is Offline Two-Player Zero-Sum Markov Game Solvable? | Simon Du | 34m |
Shows that single strategy concentration insufficient for offline two‑player zero‑sum Markov games by a simple H=1 bandit counterexample where two different… | ||
| Variants and Invariants in No-Regret Algorithms | Sylvain Sorin | 45m |
Shows that discrete no-regret procedures map to a continuous-time differential inclusion by replacing payoffs with conditional expectations and treating… | ||
| What Does Machine Learning Offer Game Theory (And Vice Versa)? | Alex Peysakhovich | 36m |
Argues that machine learning can compress big strategic environments into tractable game abstractions and shows two working patterns: for social dilemmas a… | ||
| Multi-Player Bandits With No Collisions | Mark Sellke | 41m |
Claims that exact collision-free algorithm constructions achieve minimax square-root(T) regret (up to log and poly(K) factors) with independent Bernoulli… | ||
| Global Convergence of Multi-Agent Policy Gradient in Markov Potential Games | Ioannis Panageas | 54m |
Defines a new class called Markov Potential Games and proves independent policy gradient converges to Nash policies: deterministic updates reach an ε‑Nash in… | ||
| Learning and Equilibrium Refinements | Drew Fudenberg | 59m |
Shows that when agents are long lived and derive experimentation from discounted dynamic programming, limits called patiently stable profiles must be Nash… | ||
| No-Regret Learning in Extensive-Form Games | Amy Greenwald | 56m |
Introduces Extensive-Form Regret Minimization (EFR), an algorithm that takes a chosen deviation class Φ and minimizes regret relative to Φ with complexity that… | ||
| Learning Decentralized Policies in Multiagent Systems: How to Learn Efficiently and ... | Na Li | 51m |
Proposes a Scalable Actor Critic (SAC) framework that learns fully localized policies using k‑hop truncation of Q‑functions and proves the learned policy is an… | ||
| The Role of Conventions in Adaptive Human-AI Interaction | Dorsa Sadigh | 55m |
Argues that encoding partner behavior into a learned low-dimensional latent intent Z via an autoencoder-style prediction loss and feeding it into a… | ||
| nothing matches | ||