Quality Gate for AI: How We Built an AI Control Architecture
summary
Presents an architectural approach that uses one AI as an automated ‘‘judge’’ to control the quality of another AI, illustrated with a production AI Calories Tracker that processes 5,000+ food images daily. Explains why traditional QA fails for AI, details how the evaluator AI assesses model outputs (including prompt tuning and model selection), and walks through quality-control processes across the solution lifecycle and the benefits and challenges observed in production.
AI-written summary · about
chapters (13)
- 0:00AI nutrition solutions
- 2:06Requirements and constraints
- 6:45Vendor selection and testing
- 8:09Quality Gate criteria
- 9:53Dataset creation and structure
- 11:21LLM Judge evaluation
- 14:12Google Colab test environment
- 16:40Test results analysis
- 18:35Model selection via Quality Gate
- 19:39Quality Gate in CI/CD
- 23:13Conclusion
- 25:02Q&A
- 38:05Best question awards
related talks
18mAI Quality Offensive — Content Quality without Slop and without Prompts
38mAI Just Happened to Me : Help!!
16mBuilding Applications with AI Agents
47mThe Past as a Compass for Systems Engineers in the Age of AI
22mMaking AI Accountable: Azure AI Foundry
01h 06mAI in the Trenches: Engineering Real-World Systems
32mApproach to Architecting Generative AI Solutions
29mArchitect AI success in a regulated world