Can You Walk Me Through It? Explainable SMS Phishing Detection using LLM-based Agents
summary
SmishX claims 98.8% accuracy on 1,200 real SMS messages and reports an SUS of 82.6, using LLM agents that augment chain-of-thought with URL redirect chains, domain history, HTML and page screenshots plus the top five Google results to decide phishing versus spam versus legitimate. The system generates short evidence summaries and safety advice; a user study (N=175, 40% over 65) raises accuracy from ~71% to 93% but shows overreliance: in a deliberate AI error 18 of 24 users switch to the wrong verdict.
AI-written summary · about
chapters (9)
- 0:00Explainable SMS phishing detection
- 1:59SMS phishing detection challenges
- 2:44Smish X contextual data
- 3:46Detailed and summarized explanations
- 4:29User study design and demographics
- 7:17Accuracy and usability improvements
- 8:21User disagreements and context effects
- 9:26AI mistakes and user trust
- 10:32Smish X summary and future
related talks
13mWhat Drives SMiShing Susceptibility? A U.S. Interview Study of How and Why Mobile Phone
42mThe Old is New Again: An Intro to SMS Security and Why It’s Vital
51mDeepPhish: Simulating Malicious AI
51mDeepPhish: Simulating Malicious AI
27mBlunting the Phisher's Spear: A Risk-Based Approach...
38mClick Here Now: A Positive Way to Create Phishing Awareness
41mHook, Line, & Sinker - Addressing the Human Risk of Phishing and How to Avoid the Bait
56mHacking your Mind and Emotions