~/talk

Mastering Prompts with Feedback and Pydantic

Talkat PyCon India 2025 · 2025-09-12..09-15· [ai-assisted-development, python]
name:Aarti Jha
linkedin:aartijha22
slides:none on file

A productionized prompt optimization pipeline uses LangGraph for orchestration, Olama for local model runtime, Streamlit for UI and Pydantic validation as a hard guardrail to enforce JSON schemas and reject malformed fields, with an LLM-as-a-judge multi‑metric evaluator driving an automated feedback loop. A support‑ticket extractor moves from brittle manual tweaks to iterative refinement, the headliner demo reaches target quality on the second iteration, and the system saves validated prompts once thresholds are met.

AI-written summary · about

  1. 0:00Introduction and speakers
  2. 1:11Why prompting matters today
  3. 3:45Prompting techniques overview
  4. 5:53From trial and error to engineering
  5. 7:18Building a feedback loop
  6. 8:45Evaluation framework with LLM-as-judge
  7. 9:51Introducing Pydantic for validation
  8. 11:57Pydantic validators in detail
  9. 13:29Pydantic as a guardrail
  10. 14:32System implementation overview
  11. 15:38Architecture step by step
  12. 17:04Live demo walkthrough
  13. 18:32Key learnings and takeaways
  14. 19:41Q&A session begins
  15. 20:12Handling Pydantic validation failures
  16. 21:35Avoiding overfitting in evaluation
  17. 23:00DSPy comparison discussion
  18. 24:23Setting evaluation thresholds
  19. 26:33Choosing LLM-as-judge models
  20. 27:40Pydantic vs LLM-as-judge roles
  21. 28:46Cost considerations with LLM-as-judge
  22. 29:10Ensuring reproducible LLM responses
~/search
type to search 6,518 conferences, 143,918 speakers and 192,133 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
← →
on a talk: 5 seconds back / forward
0–9
on a talk: jump to 0%–90% of the recording
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.