Verifying LLM Input and Output for Developers
summary
Practical checklist for Android apps: sanitize user text with regex and content-moderation APIs, enforce a token budget by precomputing prompt length and truncating, keep conversation history with a sliding window or summarized context, and use retrieval-augmented prompts for factual answers. Require schema validation by asking for JSON and parse it with Kotlinx.serialization, run a second LLM to perform semantic checks, post-process/redact sensitive fields, and cover happy, edge and prompt injection tests as unit and integration tests.
AI-written summary · about
chapters (19)
- 0:00Verifying LLM input/output
- 1:03Why LLM verification matters
- 2:34How LLMs work & limits
- 7:49LLM output challenges
- 9:37Two-way LLM verification
- 10:17Sanitizing LLM input
- 13:23Testing prompts in AI Studio
- 16:31Sanitizing user input
- 18:39Content moderation APIs
- 20:29Token budget limits
- 22:36Sliding windows for history
- 24:02Temporal checks
- 25:28Output consistency checks
- 26:31Post-processing LLM output
- 27:33Factual consistency with RAG
- 28:40Schema & semantic validation
- 31:06Unit & edge case testing
- 32:11Prompt injection testing
- 32:54Summary of best practices
related talks
39mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
50mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
48mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
47mValidating LLMs – A True Story
45mSecuring LLM Applications: Overcoming Security & Privacy Challenges
19mTesting LLM-Powered Applications
38mMaking LLMs reliable - A practical framework
17mModel-Based Input Validation: Preventing Prompt Injection Attacks