Preference Learning from Minimal Human Feedback for Interactive
summary
Argues robots learn objectives with far less human time by swapping demonstrations for comparisons and comparative language, using Bayesian IRL augmented with an active mutual-information query strategy that cuts required queries to about 30 queries versus ~100. Implements a shared latent space aligning trajectory encoders with a T5 encoder, augments labels with LLMs, and synthesizes many implicit preferences from one utterance. Validated in simulators, applied to exoskeleton gait tuning, and a small human study (about 10 users) shows language is faster and at least as effective.
AI-written summary · about
chapters (8)
- 0:00Human robot interaction intro
- 3:10Limitations of learning from demos
- 10:19Robot objectives from comparisons
- 17:24Binary preference comparison drawbacks
- 18:25Learning from language feedback
- 24:24Language-based robot training
- 25:48Future directions
- 26:09Cost vs performance trade-offs
related talks
10mMIND MELD: Personalized Meta-Learning for Robot-Centric Imitation Learning
14mHuman-in-the-Loop Reinforcement Learning
01h 35mDeep Robotic Learning
42mRethinking AI Alignment and Evaluation under Heterogeneous Preferences
11mLearning Socially Appropriate Robo-waiter Behaviours through Real-time User Feedback
14mTowards Unobtrusive Physical AI: Augmenting Everyday Objects with Intelligence and Robotic Moveme...
10mConditional Imitation Learning for Multi-Agent Games
01h 00mBetter Learning through Programming Languages