Breaking Boundaries with Adversarial Testing
summary
Claims that adversarial testing plus red teaming uncover real LLM failures, shows concrete exploits like Dan exploit, token smuggling and multi‑turn crescendo attack that bypass guardrails, and gives a quantifiable threat model: co‑pilot productivity estimates (3–4x, 6–10x, 20–30x, 50–80x) that accelerate risky deployment. A 1.5 million‑strong crowd supplies adversarial prompts, teams run campaigns with tools such as Pirate and PromptFu, score responses with a scoring engine, and repeatedly find failures (SQL injection, prompt jailbreaks) that red teams triage and fix.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:04AI in software development
- 3:09AI impact on testing roles
- 8:08Artificial confidence in AI
- 11:21Mr. Scam and adversarial testing
- 13:33How guardrails in LLMs work
- 14:13Email attack loophole
- 15:44Reverse psychology in LLM queries
- 16:06Dan exploit and legal loopholes
- 17:34Multi-model manipulation
- 19:01Token smuggling in prompts
- 21:14Crescendo attack exploitation
- 23:01Prompt injection in bots
- 24:28Red teaming for LLMs
- 25:56Red teaming framework
- 28:52Continuous adversarial testing
- 30:39Shift to security testing
- 32:08Key takeaways
- 32:51Q&A
related talks
59mRed Teaming Large Language Models
34mLLM Security 101 - An Introduction to AI Red Teaming
50mAdversary Emulation: How I Stopped Being Polite and Got Real Results
53mIncreasing Security ROI with Adversary Simulation and Purple Teaming
30mTeam Red: Breaking Large Language Models
46mAutomated AI Security Testing Using Adversarially Fine-Tuned AI Agents
40mFantastic Red-Team Attacks and How to Find Them
20mAI Red Teaming and Automation: Exploring Societal Risks in...