Power and Limitations of Aggregation in Compound AI Systems
summary
Presents a principal–agent model with conic feasibility constraints, feature-weights matrix α and per-agent budget levels, and proves aggregation expands the set of elicitable outputs only via three mechanisms: feasibility expansion, support expansion, and binding-set contraction, with strengthened versions giving necessary and sufficient conditions. Empirical tests use GPT‑4o Mini, GPT‑5 Mini and GPT‑5.4 on a toy paper‑list task, an LLM judge and L1 distance; literal intersection and union experiments show a persistent red vs. purple gap and that union remains useful while intersection becomes useless as model power grows.
AI-written summary · about
chapters (8)
- 0:00Aggregation in compound AI
- 2:09Why multiple LLM copies work
- 8:57Power of aggregation formalized
- 17:30Elicitability expansion mechanisms
- 25:44Binding-set contraction & support expansion
- 34:16When aggregation is useful
- 41:33Design insights for compound AI
- 44:51Q&A
related talks
16mBuilding Applications with AI Agents
01h 00mAI A-Z
01h 23mBuild your own AI Agent
44mBuilding Agentic AI Systems: From Transactions To Conversations
32mProgress In the Decentralized AI Stack
26mAI Agents and the Future of Human-AI Collaborative Intelligence
28mAgentic AI
01h 01mAI Agents In-Depth – Function Calling, MCP and Tool Use Under the Hood