Designing Production-Ready Agentic Systems: Lessons from the Field
summary
Production agents run in voice pipelines with strict latency and cost controls: time-to-first-token improves after disabling Azure OpenAI sync filters, fillers ensure acknowledgment under 700 ms, and full duplex interruption is a target. Tool-executor layers stay deterministic, text-to-SQL agents once generated queries that crashed DBs with two-hour runs so agents get deterministic budgets (example $500) and cost estimates from execution plans. Memory uses lightweight pointers and artifact-based memory for traceability; semantic cache hits reach 94% in a national deployment with hundreds of thousands of users.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 3:30Agentic system design
- 7:28Evaluation and observability
- 11:03Agentic architectures
- 15:08Tools vs agents
- 19:11Memory management
- 22:25State sharing and coordination
- 26:22Multi-agent use cases
- 28:57Cost management
- 33:16Voice assistant challenges
- 36:11Latency in voice AI
- 40:24Caching strategies
- 41:06Small models in production
- 42:12Agent communication standards
- 45:25MCP server patterns
- 47:11Semantic caching trade-offs
- 50:47Production trust and hallucinations
related talks
33mFrom Prototype to Production: Making Agentic AI Systems Reliable
59mArchitecting Microservices for Agentic AI Integration
15mEngineering Production-Grade Multi-Agent Systems in the Cloud
44mBuilding Agentic AI Systems: From Transactions To Conversations
19mNo Token Budget: Agentic Software Factories on Kubernetes
24mAI Agents in Production: Why the Platform Matters
39mFrom Demo to Production: Why Agentic AI Systems Fail - and How to Fix Them
33mEngineering AI Applications: A Conversation with AI Leaders