Markov Persuasion Process and its Reinforcement Learning
summary
Defines a Markov Persuasion Process where a planner observes the true state, a myopic actor acts on signals, and persuasion is a one-sided information control problem; planning reduces to recursive concavification of piecewise-linear value functions but pieces blow up exponentially so computing optimal policies is NP-hard in general; a polynomial case appears when receivers see full history; reinforcement learning needs a robust pessimism step and an optimism–pessimism algorithm that attains sqrt(T) regret (and √H dependence), with many open cases left.
AI-written summary · about
chapters (10)
- 0:00Introduction to MPP
- 1:25Understanding MPP Model
- 4:43Real-world Applications
- 10:37Signaling and Persuasion
- 16:52Geometric Interpretation
- 21:40Formalizing the Model
- 28:16Incentivized Exploration
- 31:42Optimal Planning
- 36:54Computational Challenges
- 39:54RL in MPP and Conclusion
related talks
01h 17mOnline Reinforcement Learning and Regret
11mLinear Predictors in Reinforcement Learning: ...
15mSequential Decision Making Problems in Multi-Agent Environments
48mPractical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...
01h 01mA Tutorial on Reinforcement Learning I
57mFundamentals of Markov Decision Processes
09mTowards Credible and Effective Data-Driven Decision-Making
52mReinforcement Learning in Recommender Systems: Some Challenges