Learning Preferences with Latent State

on YouTube ↗48m

" summary

Claims that preferences are recoverable when the human uses an unknown finite-state machine, gives a concrete estimator and proofs of consistency. Method fits TD-parameterized utilities by MLE while scoring models with prequential scoring, searches growing state-space classes, and shows convergence to a minimal machine that reproduces choice probabilities. Empirical test is a parity example (sequence length 10) where the two-state true model is recovered with little data. Assumptions: finite observations, IID choice pairs, logit noise; rare-history pathologies and model-class tuning remain open.

AI-written summary · about

" chapters (9)

1. 0:00Introduction
2. 1:04Latent state in alignment
3. 7:21Preference recovery formalized
4. 18:21Parity machine example
5. 28:37Algorithm for preferences
6. 35:07Theoretical guarantees
7. 39:03Challenges and limitations
8. 43:18Broader implications
9. 45:27Q&A

" related talks

• 44mIdentification and Estimation of Dynamic Structural Models with Unobserved ChoicesYi Xin · Information Design and Data Science 2019
• 18mConstantineConstantine · StanCon 2018
• 18mMichaelWeylandtMichaelWeylandt · StanCon 2018
• 31mStable Reinforcement Learning with Unbounded State SpaceQiaomin Xie · Reinforcement Learning from Batch Data and Simulation
• 26mAn Equivalence between Loss Functions and Non-Uniform Sampling in Experience ReplayDoina Precup · Reinforcement Learning from Batch Data and Simulation
• 31mLearning to Control Unknown SystemsRahul Jain · Societal Networks 2018
• 01h 01mA Tutorial on Reinforcement Learning IEmma Brunskill · Foundations of Machine Learning Boot Camp 2017
• 11mLinear Predictors in Reinforcement Learning: ...Andrea Zanette · Meet the Fellows Welcome Event