~/talk

Evaluating AIs: the secret recipe for not-too-dumb agents

on YouTube 22m
TalkinFrenchat Forum PHP 2025 · 2025-10-09..10-10· [ai-assisted-development]

speaker

name:Francois Zaninotto
website:https://marmelab.com/

resources

video:https://youtu.be/NjrbQ8aWDv8
slides:none on file

summary

Explains how to evaluate AI agents by defining practical metrics and tests to ensure they behave sensibly. Covers concrete evaluation techniques for agent systems (e.g., benchmark tasks, behavioural checks, and performance criteria) to detect and improve common failure modes.

AI-written summary · about

chapters (8)

  1. 0:00Introduction
  2. 3:41Why agents fail often
  3. 5:51Configuring an AI agent
  4. 8:16Evaluating agent performance
  5. 13:13Evaluation process
  6. 16:23Practical evaluation tips
  7. 18:48Key takeaways
  8. 20:39Athletics analogy

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.