But it worked in my PoC!
summary
Deploying agents to push updates across thousands of devices initially works in a five-agent PoC but fails in QA with 2 minute replies, 3M tokens per run and only 2 successes out of 30. The final system uses a hierarchical agent graph (classifier, planner, orchestrator, stage executors, dynamic spawn agents), progressive context loading, strict tool schemas, dataset IDs so agents never see raw data, deterministic DAG-like execution with tool chaining and a scheduler for long waits, full observability — and that architecture ships and scales.
AI-written summary · about
chapters (7)
- 0:00Agentic framework for updates
- 3:46PoC failure in QA
- 9:59Redesigning agent architecture
- 16:50Context engineering and determinism
- 21:38Data processing isolation
- 24:29Long sessions and retries
- 25:33Final architecture and observability
related talks
32mWhat Agents Do When You’re Not Looking
22mThe Agent Control Plane: Managing Thousands of AI Agents
24mAI Agent Swarms Are Amazing
40mHow To Tame Your Agents?
07h 47mAI Agents Conference 2026 - Day 5
46mWhen You Run Out of Requirements: What Happens When You Go All-In on Agents
45mAI agents in practice - beyond the prototype
40mFrom Prompt to Multi Agent System: The evolution of our AI product