QA for ML: How we can trust AI with Food Sustainability
summary
Claims that QA tames AI get grounded in specifics: a 40-line chatbot prototype built with Claude fails factual checks and prompts error analysis; retrieval and grounding use retrieval-augmented generation plus internal prediction APIs; observability shows models degrade and are stochastic so the same prompt run repeatedly yields variance so tests run with 20 variations and repeated-query statistics; human evaluation via Amazon Bedrock ranks outputs; guardrails, metrics and automated plus human loops reduce hallucinations though stability still requires continuous retraining and monitoring.
AI-written summary · about
chapters (13)
- 0:00Trusting AI in food sustainability
- 1:07Why quality matters for AI
- 2:50What is QA in software
- 5:16QA in software vs AI
- 9:35AI vs traditional software
- 15:13MLOps and ML testing
- 18:30Explaining AI models
- 20:15AI in agriculture use cases
- 27:32Trustworthy AI chatbots
- 33:40Limitations of large language models
- 35:47Grounded AI response tools
- 38:35AI reliability testing
- 40:00Human-in-the-loop QA
related talks
29mTesting for Ghosts in the Machine: Assuring 'Good Enough' Software Quality in AI-based systems
15mChallenges in moving Gen AI apps to production
13mNew Challenges in Software Quality Brought by GenAI and LLMs
41mAutonomous Quality Engineering: AI Systems That Generate Execute Heal Learn and Govern Quality
32mQuality engineering in the AI age: From testing AI to leveraging GenAI
34mTechnical: Harvesting Trust: Methods for Validating Chatbots in
01h 04mWhat are we thinking in the age of AI?
49mHype and Reality: How to Evaluate AI/ML in Cybersecurity