Robust NLP Models for real-world deployment
summary
Sandya Mannarswamy discusses why NLP models often fail to generalize in production and emphasizes rigorous evaluation across diverse inputs (including adversarial examples) to reveal biases and weaknesses. The talk covers current research and industry best practices for evaluating model generalizability and robustness, and gives practical tips for preparing NLP models for real-world deployment.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:17Context
- 6:43Building a NLP Model - Current Recipe
- 7:32NLP Model Development Cycle
- 8:07Real World Data Can Be Highly Diverse
- 10:40Current NLP Paradigm
- 13:11Machine Reading Comprehension
- 13:50Did the Machine Understand the Question?
- 18:00Argument Reasoning Comprehension
- 20:22Improving Model Robin
- 26:32Dataset Ablation Analysis
- 28:08Preventing Models from Learning Surface Cues
related talks
01h 16mGeneralization in natural language processing
55mKeynote - Natural Language Processing: Challenges and Next Frontiers
31mRemaining Challenges in Deep Learning based NLP
59mWhy Great Models Fail: Lessons From 9 Years of Deploying ML Models
17mNatural Language Processing in Automation
06mAdvancements in Natural Language Processing (NLP) and Text Analytics
15mChallenges in moving Gen AI apps to production
39mHow to Avoid structural biases in evaluating your Machine Learning/NLP projects