Automated AI Security Testing Using Adversarially Fine-Tuned AI Agents
summary
Claims that adversarially fine‑tuned agents can run millions of meaningful attacks are backed by a concrete pipeline: use Garak and Pirate‑style test runners, obliterate alignment on a base model like GPT‑4 Omni, fine‑tune with synthetic prompt‑injection datasets, and validate against a live target such as Llama 3.1 hosted on VLM. A pentest LLM is built and trained with an inline reward that checks live jailbreak success within ten turns, final runs cost ~USD 300 and prove the approach works but needs per‑target data and continuous testing.
AI-written summary · about
chapters (8)
- 0:00AI security testing challenges
- 3:36AI systems and security risks
- 10:46Current AI testing limits
- 19:44Automated red teaming concept
- 29:21Fine-tuning adversarial models
- 36:32Insights from red teaming experiments
- 38:40Conclusion and recommendations
- 40:28Q&A
related talks
46mScalably Understanding AI with AI
01h 24mOffensive Cybersecurity by AI Promises and Pitfalls
45mSecuring AI: There Is No Try, Only Do!
27mKeynote: Towards Building Safe and Secure Agentic AI
19mKeynote: Towards Building Safe and Secure Agentic AI
34mLLM Security 101 - An Introduction to AI Red Teaming
53mHow to Break AI Systems (Before Someone Else Does)
43mSecurity of GenAI & LLMs: A New Era of Ethical Hacking