The Unreasonable Effectiveness of Prompt Learning
summary
Claims a feedbacK-driven system-prompt learning loop that applies RL-style updates to prompts not weights and auto-tunes an agents.md. Uses unit tests + LLM-as-judge evals to generate English explanations, feeds those into a meta-prompt to diff and append rules, and runs on SWEbench (150 examples) and BBH. Tests on Claude and Klein yield ~+5% (cloud code) and ~+15% (client) more GitHub issues resolved and fewer rollouts than DSPI GEA.
AI-written summary · about
chapters (9)
- 0:00Prompt learning for coding agents
- 1:18System prompt learning importance
- 2:19RL vs prompt learning for agents
- 4:06Benchmarking coding agents
- 5:28Prompt learning process
- 7:14LLM-as-judge evaluations
- 8:36Results of prompt learning
- 9:18Prompt learning vs GEA
- 10:23Closing thoughts
related talks
52mBuild a Prompt Learning Loop
30mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
14mPrompt Engineering is Dead
13mPrompt Engineering Simplified
02h 01mPrompt Engineering and AI Red Teaming
30mIs Prompt Engineering Dead?
14mUnconference: Self-optimizing deep research agents
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI