Learning Decentralized Policies in Multiagent Systems: How to Learn Efficiently and ...
summary
Proposes a Scalable Actor Critic (SAC) framework that learns fully localized policies using k‑hop truncation of Q‑functions and proves the learned policy is an O(ρ^κ)-approximation to a first‑order stationary point of the global objective; shows exponential decay of distant influence (ρ<1, empirically much smaller than γ) justifies local learning; proves two‑timescale actor‑critic converges to a neighborhood whose radius is the truncation error; observes stationary points coincide with Nash equilibria even under identical interests.
AI-written summary · about
chapters (18)
- 0:00Intro
- 2:38Opportunities and Challenges Decision-making
- 3:50Learning (Data-driven decision-making) is a promis
- 5:37Control of Networked Markov Decision Process
- 9:29Examples of Systems with the local interact
- 12:26Scalable RL for Network Systems
- 13:46Review: Policy Gradient in the Full Information C
- 15:12RL in the Network Setting
- 19:16The Exponential Decay Property
- 25:27Truncation of Q-function
- 32:05Numerical results: Multi-Access Wireless Communic
- 32:53Other (Multiagent) Learning Settings Decentralized Control
- 33:56Optimality Guarantee
- 34:50Optimization Landscape
- 39:42Gradient play for identical interest case
- 41:41General Stochastic Games
- 42:46Convergence of gradient play?
- 49:14Summary
related talks
38mRetrospective on multi-agent learning as a path to intelligence
32mVirtual talk by Janardhan Kulkarni
14mUnconference: Self-optimizing deep research agents
19mTraining Agentic Reasoners
21mUsing Deep Reinforcement Learning for Dialogue Systems
50mScalable Innovation: Exploring Multi-Agent Systems
45mPanel Discussion
53mIterative preference learning methods for large language model post training