Secure by Design: Strategies for LLM Adoption in Cloud-Native Environm...
summary
Built a cloud‑native RAG/agent platform, aligned threat modeling to OWASP Top 10 for LLMs and runs garak and Gizard scans against RAG apps using the Tesla manual as a KB. Finds prompt hardening is cheapest and often most effective; small models (Llama 3 8B) with defenses beat large models without them; GPT family shows higher innate resistance. NeMo Guardrails triples latency and cost; Llama guard adds ~300 ms. Scans take hours, show 20–40% false positives, and single scans can cost from a few dollars to thousands at scale.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 0:50Agenda
- 1:57Our teams
- 2:10Our AI development platform
- 4:37Threats in our systems
- 5:40Multiagent system
- 6:48Objectives
- 7:21Offensive Practices
- 11:18Test Scenario
- 12:14Gizcard
- 12:51Prompt injection
- 13:51Hallucination
- 14:36Sensitive Information Disclosure
- 15:25garak
- 16:24conclusion
- 18:41performance
- 20:54cost
- 23:37conclusions
- 26:48future directions
- 28:37QA
related talks
46mGuarding the LLM Galaxy: Security, Privacy, and Guardrails in the...
47mA Step-by-Step Guide to Securing Large Language Models (LLMs)
50mSecuring LLM-Powered Applications: Overcoming Security and Privacy Challenges
29mBeyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails
47mHarnessing LLMs for Next-Level Security Analysis
26mOne guardrail is not enough: Defense-in-depth against LLM jailbreaks
44mSecuring LLM-Powered Apps: Overcoming Security and Privacy Challenges
20mSecurity Threats in Modern LLM Applications