Sequential Decision Making Problems in Multi-Agent Environments

on YouTube ↗15m

" summary

Group presents a method called factored policy gradient that uses an influence network (bipartite graph linking action dimensions to reward components) and a factored baseline to attribute credit only to responsible policy factors. Paper proves these baselines are valid under an unbiased influence model, open-sources two packages, and shows experiments: a search-bandit toy and a traffic-grid control problem where localized influence assumptions speed learning and outperform joint policies while action-dependent baselines take two orders of magnitude more wall-clock time.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 4:00Motivation
3. 5:53Markup Decision Process
4. 7:31Influence Network
5. 8:45Policy Factorization
6. 10:39Factored Baseline
7. 12:45Traffic Optimization

" related talks

• 37mMultiagent Reinforcement Learning: Rollout and Policy IterationDimitri Bertsekas · Reinforcement Learning from Batch Data and Simulation
• 30mLearning Multi-Agent Collaborations With DecompositionYuandong Tian · Reinforcement Learning from Batch Data and Simulation
• 21mScalable agent-based models for optimized policy design: applications to the ec...Sharan Agrawal · PROPL 2024
• 15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust ControlKaiqing Zhang · Meet the Fellows Welcome Event
• 42mA Regret Minimization Approach to Mutli-Agent Control and RLElad Hazan · Multi-Agent Reinforcement Learning and Bandit Learning
• 10mConditional Imitation Learning for Multi-Agent GamesDorsa Sadigh, Stefano Ermon, Andy Shih · HRI 2022
• 45mPolicy Gradient: Optimal Estimation, Convergence, and Generalization beyond Cumulative RewardsMengdi Wang · Adversarial Approaches in Machine Learning 2022
• 42mReinforcement Learning using Generative Models for Continuous State and Action Space SystemsRahul Jain · Reinforcement Learning from Batch Data and Simulation