Harnessing AI for Malevolence: A Hackers Tale
summary
Live demos show prompt injection bypasses 15 of 20 online summarizers tested, including an email assistant coerced to add a malicious “evil.com” link and speak like a 15th century pirate. Automated hallucination creates fake npm packages (reported 20–60% hallucination rate) enabling hallucination squatting where attackers publish those names. Malicious models on Hugging Face (example: White Rabbit) and tools from Lassu Security are used to refactor old exploits and scale phishing, while agents that sanitize prompts and outputs are the best-mitigation approach available.
AI-written summary · about
chapters (9)
- 0:00AI-driven security risks
- 1:12Prompt injection attacks
- 3:38Jailbreaking AI controls
- 6:05Malicious AI in email assistants
- 8:31AI hallucinations and package risks
- 11:22Hallucination squatting in open source
- 12:46Malicious AI models
- 13:49Refactoring exploits with AI
- 14:52AI security challenges
related talks
51mPrompt Injection: When Hackers Befriend Your AI
40mThe Mechanisms That Enable Misuse and What We Can Do About It
33mAgentic ProbLLMs: Exploiting AI Computer-Use and Coding Agents
36mAgentic AI Under Attack: Live Demos of Exploits Through Autonomy and Trust
26mHow can we abuse AI hallucinations to feed the already bloated software supply chain
43mThe AI Survival Guide: Practical Advice for Stressed-out Security Teams
01h 01mExploiting the supply chain
31mHacking AI Applications