How to Break AI Systems (Before Someone Else Does)
summary
Large models fail because they treat instructions as data and are trivially jailbreakable; demos include hidden prompts in documents, agent goal manipulation, and data‑exfiltration via emails and PDFs. Concrete impacts: a Bard demo triggers a 9% stock drop (~$100B market cap) and a chatbot error leads to legal liability over airfare refunds. Exploits shown include the Crescendo attack and simple encodings; defenses are practical, e.g. spotlighting with base64 encoding, which cuts attack success from ~35% to ~3% on GPT‑3.5 while barely impacting GPT‑4, and a red‑teaming practice platform with vulnerable apps is provided.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:39AI red teaming
- 10:07Architectural flaws in AI models
- 12:57Prompt injection attacks
- 17:51Structural red teaming
- 19:37Indirect prompt injection risks
- 22:30AI misuse scenarios
- 27:29Nation-state threats
- 30:16Mitigations and spotlighting techniques
- 36:14Q&A
related talks
53mHow to Break AI Systems (Before Someone Else Does)
01h 01mHow to Break AI Systems (Before Someone Else Does)
37mKeynote - Hacking AI How to Survive the AI Uprising
30mBattling the Machine: Cutting-Edge Attacks and Defenses in Generative AI
45mSecuring AI: There Is No Try, Only Do!
51mPrompt Injection: When Hackers Befriend Your AI
35mHacking AI Systems: How to (Still) Trick Artificial Intelligence
44mHacking AI - How to Survive the AI Uprising