Listen, Attend, and Walk : Interpreting natural language navigational instructions
summary
Presents an encoder–decoder (sequence-to-sequence) deep learning approach with alignment (attention) to map natural-language route instructions to action sequences for a navigation agent. Covers data preprocessing, training, evaluation and simulation in a virtual environment, and discusses design challenges and trade-offs; the model learns word meanings, spatial relations and compositional semantics from data without hand-crafted linguistic features.
AI-written summary · about
chapters (14)
- 0:00Start
- 3:01For instance...
- 3:56Introduction
- 7:34Steps involved
- 7:58Get data
- 8:20Transform
- 8:35Convert NL instruction to vector
- 9:47Encoder-Decoder Model
- 10:31Overall Architecture
- 15:03Multi-level Aligner
- 19:02Attention Decoder
- 21:12Train the model
- 22:54Training Vs. Validation error
- 23:25Test the model
related talks
40mTeaching a Car to Drive Itself by Imitation and Imagination
26mUsing deep learning in natural language processing
01h 31mNatural Language Understanding: Foundations and State-of-the-Art
24mMulti-Modal AI Agents Explained | Open-Vocabulary Alignment
49mDeep Learning Applied to Natural Language Processing (NLP)
36mVibing with Data -- Multi-Agent Data Modeling and Constructi...
24mQuoc Le, Software Engineer, Google
42mAttention and Memory in Deep Learning Networks