Making AI Accountable: Azure AI Foundry
summary
Demo shows Azure AI Foundry running an evaluation SDK and a GitHub Action to catch regressions in CI/CD and production. It measures coherence, fluency, groundness, document retrieval, intent resolution and task adherence, flags safety issues and resource cost, and surfaces explanations for scores. Example: an eShop agent scores high while a misused banking agent fails relevance and task adherence, letting engineers pinpoint prompt, KB or tool‑calling failures.
AI-written summary · about
chapters (9)
- 0:00Introduction to AI Accountability
- 0:17Personal Introduction and Background
- 0:59Challenges in AI Observability
- 2:40Moving from Magic to Measurable
- 3:24Microsoft's AI Foundry Tools
- 3:39Evaluators for AI Applications
- 7:53Custom Evaluators and Use Cases
- 9:53Demo: AI Foundry in Action
- 17:43Q&A and Final Thoughts
related talks
20mAgentic Excellence: Mastering AI Agent Evals w/ Azure AI Evaluation SDK
43mOperationalize AI responsibly with Azure AI Studio
16mResponsible AI at Scale
24mHow to Test AI Features Before Your Users Do
01h 54mBuilding Code First AI Agents with Azure AI Agent Service
16mLeveraging Azure AI Foundry to Build Intelligent Release Agents
36mBuilding AI Agents in .NET with Azure AI Foundry
11mBuilding Careers in AI