Context is all you need: FUNdamental linguistics for NLP
summary
Argues that distributional hypothesis explains much of LLM competence, shows concrete tools—alotaxonometer, shiftator, labMT lexicon and Python packages—that compare rank distributions, track sentiment time series (Frankenstein example with clear end‑game dips), and cluster token vectors to reveal fine‑grained senses; demonstrates BPE tokenization tradeoffs; reports that tools produce interpretable fingerprints and useful diagnostics but that models lack direct grounding so context, tokenization, corpus choice and reference selection materially change outcomes and claims about truth.
AI-written summary · about
chapters (13)
- 0:00Linguistics and NLP intro
- 1:45Human language origins
- 5:37Morphology for NLP
- 9:36Words as symbols
- 11:45Syntax and meaning
- 12:50Morphological complexity
- 15:59Allotaxonometer analysis
- 19:30Text as time series
- 26:31Language change semantics
- 31:10Meaning construction
- 36:54Symbol grounding problem
- 40:50Pragmatics and context
- 44:52Responsible NLP
related talks
01h 31mNatural Language Understanding: Foundations and State-of-the-Art
45mReasoning in Natural Language
01h 26mBagging to BERT- A tour of applied NLP
17mLow-Resource Languages as Stress Tests for NLP Data
31mRemaining Challenges in Deep Learning based NLP
01h 16mGeneralization in natural language processing
43mThe Brain’s Guide to Dealing with Context in Language Understanding
55mDemystifying Natural Language Processing across several industry verticals