Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments...
summary
Automated text-matching used for reviewer assignment is exploitable: combining adversarial archive curation (keeping only the most similar past paper) with adversarial abstract modification (one added sentence plus up to 10 keyword inserts) raises a reviewer from rank ~1000 to top-1 about 50% of the time on NeurIPS-era data. Attacks target Specter embeddings and max- or avg-pooling of cosine similarities. Defenses that work: force larger reviewer archives, use average or 75th-percentile pooling, and tune on prior-year public reviewer pools. Artifacts and examples are published.
AI-written summary · about
chapters (12)
- 0:00Collusion risks in assignments
- 2:42Collusion rings impact
- 4:50Text-matching vulnerabilities
- 5:13Goals of colluding reviewers
- 6:39Text similarity scores
- 7:01Adversarial archive curation
- 8:26Adversarial abstract modification
- 9:50Tuning attacks with past data
- 11:15Collusion attacks effectiveness
- 12:38Defense mechanisms
- 13:40Future directions
- 14:45Practical impact
related talks
36mAdversarial Machine Learning: How Secure Is ML?
46mA Short Introduction to Hacking Artificial Intelligence
35mHacking AI Systems: How to (Still) Trick Artificial Intelligence
25mPDF Mirage: Content Masking Attack Against Information-Based Online Services
31mAttacking Deep Learning-Based NLP Systems
53mSecurity of Machine Learning Systems
44mComputers are Stupid: Protecting "AI" from Itself
28mSecurity of Machine Learning