Corruption Robust Exploration in Episodic Reinforcement Learning
summary
Proposes algorithms for episodic RL under an adversarial corruptions model where an unknown number C of whole episodes can be arbitrarily altered; main guarantee is near‑optimal stochastic regret multiplied by a factor scaling linearly in C (tabular) and analogous bounds for linear MDPs (dependence on number of actions remains). Method combines optimistic value iteration with computed lower bounds, active sets and an ensemble of base learners that randomly switch using local and global data to tolerate non‑i.i.d. transitions. Open problem: reduce the linear dependence on C.
AI-written summary · about
chapters (9)
- 0:00Intro
- 0:30Prelude
- 3:00Model
- 8:38Prior work
- 10:40Active sets and optimism
- 15:08Full algorithm
- 16:54Analysis
- 19:52General Framework
- 21:28Extensions Open Questions
related talks
12mDistribution-Free Robustness
38mBest of Both World Algorithms from I.I.D. to Adversarial Data
01h 15mBridging Stochastic and Adversarial Bandits
01h 17mOnline Reinforcement Learning and Regret
46mCorruption-Robust Contextual Search
11mLinear Predictors in Reinforcement Learning: ...
34mOffline Reinforcement Learning and Model-Based Optimization
15mMulti-Agent Reinforcement Learning In Stochastic Games: From Alphago To Robust Control