Reward-Guided Synthesis of Intelligent Agents with Control Structures
summary
ReGuS directly synthesizes readable control programs from sparse reward signals using a hierarchical algorithm that decomposes loop search, performs on-demand synthesis of conditionals, and adds learned procedures via curriculum synthesis. It compresses exploration for long-horizon tasks (≈1,000 steps) by nesting loops and reusing skills, builds working controllers for navigation, driving, locomotion and pick-and-place, and shows much better sample efficiency than deep RL and program-synthesis baselines — e.g., early sketches score 0.1 by picking 1 of 10 markers.
AI-written summary · about
chapters (11)
- 0:00Reward-guided synthesis intro
- 2:32Deep RL limitations
- 4:19RIGuS framework
- 5:43Domain-specific language for control
- 7:11Sparse rewards handling
- 8:14Sketch generation
- 10:03Conditional synthesis
- 12:09Curriculum synthesis
- 13:35Experimental results
- 16:10Conclusion
- 16:31Q&A
related talks
15mAbstraction Refinement-guided Program Synthesis for Robot Learning from Demonstrations
24mRecursive Coding Agents
26mCan Reinforcement Learning Lead to AGI?
01h 00mBetter Learning through Programming Languages
01h 10mNeurosymbolic Program Synthesis: Bridging Perception and Reasoning in Real-World(…)
15mAgentic Reasonings with Reinforcement Learning
01h 17mTraining intelligent game agents using deep reinforcement learning
42mMaking state management intelligent