How we taught agents to use good retrieval
summary
The talk claims Mixedbread closes the Oracle gap by teaching agents to choose among four real tools — overview search, semantic search, filter chunks and grep tool — and trains a small LLM with supervised fine-tuning plus on‑policy RL using a retrieval reward (NDCG plus LLM judging) and a trajectory reward. Benchmarks: mixed results recover most Oracle loss (BrowseComp gap falls to ~3 points, Office QA Pro nearly closed), Oblique NDCG@10 = 0.40 vs GPT multi‑hop 0.18, MatchQA accuracy 93.4 with Gemini 3.5 flash.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:06Reasoning versus retrieval gap
- 3:15Benchmarking retrieval gap
- 5:41Search agent for knowledge work
- 8:33Agent harness and tools
- 10:22Training search agent with RL
- 13:17Results and future directions
related talks
22mAgentic Hybrid Retrieval: Orchestrating LLM Agents, Hybrid Search & Explainability in OpenSearch
01h 30mOptimizing Retrieval for Agentic Systems
46mArchitecting Efficient Agents with Semantic Routing and Mixture-of-Tools
37mRAG, Tooling, MCP, Agents ... and What is Missing?
36mBeyond Grep: Search for Reliable Coding Agents
01h 00mLLMs gone wild
40mJudgement Day: Benchmarking "Black Box" LLMs With Open Legal Datasets
28mNavigating Challenges and Enhancing Performance of LLM based Applications