Of Moments and Matching: Trade-offs and Treatments in Imitation Learning
summary
Presents a unifying game‑theoretic recipe that casts three imitation regimes as minimax moment‑matching games (off‑Q, reward‑matching, on‑Q), proves that approximate equilibria give imitation‑gap bounds, and identifies the familiar compounding rates: behavioral cloning can incur epsilon T^2 regret while interactive correction yields epsilon T. Uses the performance‑difference lemma, occupancy‑measure reductions, no‑regret dynamics and planning oracles; offers three practical algorithms (Advil / Advil / Zadquil) that beat baselines on tasks, while noting open statistical, realizability and computational gaps.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 3:48Our contribution
- 7:31Behavioral Cloning
- 11:38Data Set Aggregation
- 13:28Interactive Expert
- 14:33Epsilon
- 16:26Absolute T squared
- 17:46Interactive experts
- 18:33Three paradigms
- 20:31Goal
- 22:17Imputation Gap
- 26:39OffCue Game
- 30:10Minimax formulation
- 31:18Approximate equilibrium
- 38:42Minimax games
- 42:32Summary
- 45:17Future work
related talks
04mOn gradient-based methods for finding game-theoretic equilibria
10mConditional Imitation Learning for Multi-Agent Games
01h 17mOnline Reinforcement Learning and Regret
55mLearning Minimax Estimators Via Online Learning
48mPractical Model-based Algorithms for Reinforcement Learning and Imitation Learning, with...
56mA Tutorial on Reinforcement Learning II
58mMemory-Regret Tradeoff for Online Learning
01h 11mA Theory of Multi-objective Machine Learning