Building and Testing AI-Agent Powered LLM Applications: A Live Demonstration
summary
Live demo builds an agentic RAG system that combines chunked PDF vectors (stored via llama‑index) with a Google agent (node google + SerpAPI) to fetch GitHub results, synthesizes them with GPT‑4 and returns answers with citations. Tests run with a critic LLM measure faithfulness, correctness, relevancy and other metrics (latency, token count, source hits); thresholds are configurable (example used: 70% threshold). A query lacking PDF data triggers Google retrieval and synthesis; evaluation shows some responses score low, confirming the pipeline and observability work end to end.
AI-written summary · about
chapters (10)
- 0:00Welcome
- 2:36RAG importance
- 8:41Agentic systems need
- 15:51Live RAG demo with PDF
- 27:19Evaluating LLM metrics
- 33:45Testing RAG agents
- 38:44Agents in real use
- 40:49Q&A on reliability
- 44:40Bias and ethics testing
- 46:51Closing
related talks
02h 08mIntroduction to AI Agents
18mLLMs and GenAI: A Guide to Building Your Own RAG Model for Document Interaction
27mAI Demystified Unveiling the Intelligence Behind the Machine
28mAPIs and GenAI : Your Perfect Combo with LLM-Powered Frameworks.
36mTechniques to avoid common LLM pitfalls
01h 00mLLMs gone wild
41mBuilding LLM Agents – Challenges and Learnings
32mBuilding Multi-Agent RAG Systems