Increasing the Analysis Surface of Large Language Models
summary
Claim: model internals reveal attack signals that black‑box filters miss. The team develops attention-based analysis linking unsafe prompts to attention patterns, trains a classifier on five open‑source malicious prompt datasets and reports 98% / 85% / 98% / 87% / 78% detection on Mistral‑7B. They integrate the method into an Olama front end / Fort wrapper that blocks jailbreaks and math‑encoded exploits without input/output filtering. Work is NIST‑funded (SBIR) and presented as a complementary mitigation, not a complete solution.
AI-written summary · about
chapters (12)
- 0:00Introduction and speakers
- 1:50Agenda and transformer basics
- 5:26Human vs transformer thinking
- 11:05LLMs in cyber security
- 18:22Limitations of input-output filters
- 21:12Semantic leakage and attacks
- 23:47Analyzing model internals
- 25:34White box analysis demo
- 30:10Fort wrapper in action
- 33:01Documentation and model cards
- 35:28Key takeaways and future work
- 38:37Q&A session
related talks
01h 01mIntroduction to AI Security
26mOne guardrail is not enough: Defense-in-depth against LLM jailbreaks
59mRed Teaming Large Language Models
30mTeam Red: Breaking Large Language Models
23mSecurity and auditing tools in Large Language Models (LLM)
26mWhen Models Write Malware: Defending Against On-demand Exploits
34mHacking the Blackbox
42mHuman in the loop is not enough