The Dark Side of AI: The Hidden Risks in Open-Source AI Models
summary
Covers concrete attacks and demos: a popular npm library with 10 million weekly downloads becomes a dropper, cncjs pulls in 800 transitive packages, and typosquatting and star-jacking create packages that mirror 9,000+ star repos. Shows how pickled Hugging Face models enable pickle RCE and how a poisoned code‑completion “Malicious Copilot” IDE plugin injects spaced payloads that open reverse shells. Recommends using pickle-scan, avoiding unvetted models, and treating popular packages with suspicion.
AI-written summary · about
chapters (22)
- 0:00Software supply chain threats
- 1:25Open source dominance
- 2:06NPM account hijacking
- 3:31Geolocation malware
- 5:17Star jacking explained
- 7:23Typosquatting attacks
- 8:48Clipboard hijacking demo
- 9:30Remote shell attacks
- 11:18Discord malware in npm
- 12:23Ly Gang attack group
- 13:28LinkedIn job scams
- 15:16AI and generative tools
- 16:41Hugging Face risks
- 18:07Python pickle risks
- 19:32Malicious AI model demo
- 21:28Model poisoning
- 23:13Key takeaways and mitigation
- 24:41Q&A static analysis
- 26:49Developer behavioral takeaways
- 28:14Q&A attack detection
- 29:42Trusting AI models
- 31:08Final Q&A
related talks
56mHidden Risks in Open-Source Code and AI Models
45mThe Dark Side of AI: The Hidden Supply Chain Risks in Open...
26mSmarter Code, Sneakier Risks: Supply Chain Security in the Age of AI
28mOwn Goal: How AI Agents Are Quietly Scoring for the Other Side
30mThe Simple, Yet Lethal, Anatomy of an AI Attack
52mSleeping With One AI Open: An Introduction to Attacks Against...
05mIgnite: Security of Open-Source AI: is there any difference?
52mAI Is Just Software, What Could Possibly Go Wrong?