One Poisoned Artifact Can Steer Your AI: How Robust Are Your LLM-Assisted Security Workflows?
summary
Researchers build a RAG pipeline with BC embedding base as retriever and qwen 34B as generator and show a single attacker‑crafted document containing a retriever bait and a malicious instruction shifts attention so the document moves from rank ~15 of 500 to rank 2 and downstream attack success rises from ~18% to as high as 86% after generator optimization. A live end‑to‑end demo returns a command with an attacker URL and proposed defenses include context sanitization, guardrail models, and flagging semantic outliers.
AI-written summary · about
chapters (11)
- 0:00AI security risks
- 3:13What is RAG and why
- 8:49Enterprise risks of poisoned artifacts
- 11:40RAG for security workflows
- 17:21Designing poisoned document attacks
- 22:18Exploiting attention mechanisms
- 28:21Live demo setup
- 32:12Executing the poisoning attack
- 35:25Improving attack sophistication
- 36:52Key findings
- 39:41Q&A
related talks
26mOne guardrail is not enough: Defense-in-depth against LLM jailbreaks
26mBreaking RAG Systems: Exploiting Vulnerabilities & Hardening Your GenAI Applicatio...
33mSecuring Retrieval-Augmented Generation
26mWhen AI Gets Manipulated: Security Risks in LLM Applications
29mOne Bad OSINT Can Ruin Everything: How Secure is Your CTI RAG System, and Can You Minimize the Risk?
51mFrom Text to Flaws: vulnerabilities in applications with Generative AI and LLMs
36mAgentic AI Under Attack: Live Demos of Exploits Through Autonomy and Trust
57mLLMs gone wild