Neural Networks for Natural Language Processing
summary
Covers concrete experiments: German dataset with 300‑dim fast text embeddings fed into feed‑forward, LSTM and transformer models implemented in PyTorch, truncating to ~100–225 tokens. Optimizer and scheduler choices materially change training dynamics; fine‑tuning the whole pre‑trained model outperforms training only a linear head. Reported results reach around 85% accuracy for a 2‑layer LSTM (longer/deeper models cost more GPU time), and the fasttext classifier is noted as working well in production.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:47Neural network basics
- 5:40Feed-forward NN in PyTorch
- 10:45Optimizers and schedulers
- 15:14Training from scratch
- 19:58Text to word vectors
- 28:41Variable-length text input
- 33:05Word embeddings limits
- 40:46RNNs and LSTM cells
- 51:30LSTM model training
- 59:28LSTM benchmarking
- 1:06:23Sequence-to-sequence learning and attention
- 1:11:00Contextual embeddings
- 1:18:30Transformer architecture
- 1:24:05Subword tokenization
- 1:28:06Fine-tuning transformers
- 1:30:37Summary and Q&A
related talks
45mDeep Learning for Natural Language Processing
01h 40mA Deep Dive into NLP with PyTorch
38mFrom Von Neumann to Transformers: An Infra Engineer's Guide to What's Actually Inside an LLM.
01h 30mIntroduction to Deep Learning & Natural Language Processing
25mGenerative Text Modelling: Scratching the surface
23mUnveiling the Journey of NLP
44mTransfer learning in NLP
12mRevolutionizing Cloud-Native Applications with NLP