Practical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...
summary
Proposes a model-based RL recipe that learns neural-network dynamics with a reward-aware loss (no squared error), plugs that model into a planner solved by standard model-free policy search (e.g. TRPO), and enforces an optimism-style constraint via an upper bound on Vπ(M) with a KL closeness to a reference policy. The algorithm SLBO gets monotone improvement guarantees and polynomial sample complexity under oracle assumptions and empirically beats model-free baselines (~1M vs ~10M MuJoCo steps) while sometimes predicting nonphysical states early.
AI-written summary · about
chapters (13)
- 0:00Introduction to RL challenges
- 2:50RL fundamentals and definitions
- 7:31Model-based RL overview
- 11:21Research motivation and goals
- 15:05The exploration vs exploitation
- 18:47Challenges in measuring error
- 23:01Developing a consistent measure
- 26:50Proposed algorithms and theory
- 31:27Bounding performance differences
- 36:12Improving practical implementation
- 39:26Experimental results and demo
- 42:36Extensions and conclusions
- 43:57Open questions and Q&A
related talks
34mOffline Reinforcement Learning and Model-Based Optimization
54mGeneralization and Robustness in Offline Reinforcement Learning
01h 17mOnline Reinforcement Learning and Regret
42mReinforcement Learning using Generative Models for Continuous State and Action Space Systems
15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust Control
01h 11mFoundations of RL
11mLinear Predictors in Reinforcement Learning: ...
01h 01mA Tutorial on Reinforcement Learning I