From Deep Research to Evaluation and Optimization of Advanced AI Age...
summary
A modular Deep Research agent is built with planner extractor merger writer stages that call hybrid search (BM25 + vectors + rerankers) and RAG on private data; synthetic training data (INPAR) and an open-source ELO scorer (Ragilo) implement LLM-as-a-judge to rank outputs without gold labels. Prompt-optimization experiments on a 100-question CS benchmark use 4.1-mini and a DeepResearch search API, showing GPA outperforms TextGrad (≈41 vs >60 tries), about $30 per full run, and meaningful gains from simple prompts.
AI-written summary · about
chapters (10)
- 0:00Advanced AI agents intro
- 3:38RAG pipelines evolution
- 7:11Evaluating deep research outputs
- 10:02Deep research agent architecture
- 14:16Optimizing multi-agent prompts
- 17:30Benchmarking prompt optimization
- 23:10Manual prompting outdated
- 26:43Debugging agentic systems
- 29:10Retrieval vs agentic optimization
- 31:42Q&A
related talks
14mUnconference: Self-optimizing deep research agents
01h 57mFull Workshop: Build Your Own Deep Research Agents
40mHow [not] to evaluate your RAG
43mHarnessing DeepSeek-R1 and OpenSearch for RAG-Powered Intelligent Info...
40mAgentic Retrieval: Building Self-Optimizing Search Systems
13mThe State of AI Powered Search and Retrieval
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production
44mApropos DeepSeek