Efficient Transfer Learning
summary
Booking.com’s MT pipeline uses OpenNMT-tf on TensorFlow with multi‑GPU training to scale from in‑house property descriptions to up to 96 million parallel sentences. Domain transfer raises negative‑review quality from ~79% acceptable to ~90.4% after multi‑domain training; cross‑lingual transfer (initialize from Russian, or joint‑train many languages) fixes long‑tail Ukrainian errors. Two practical methods are shown: transfer via weight initialization versus slower but stronger joint training.
AI-written summary · about
chapters (15)
- 0:00Booking goals
- 1:03What is transfer learning
- 2:28User-generated content challenge
- 4:01Multi-domain training
- 6:28Translation model results
- 7:56Sub-word tokenization
- 9:47OpenNMT and transformer
- 11:12Live demo: English-Spanish
- 13:20Pitfalls of transfer learning
- 14:47Low-resource language challenges
- 17:34Related language weights
- 19:23One-to-one transfer limits
- 19:44Joint learning for tasks
- 21:26Joint training results
- 23:13Summary
related talks
44mTransfer learning in NLP
45mTransfer Learning - Entering a new era in NLP
36mScaling Data Beyond English: Multilingual Scaling and Data Mixing Laws
40mHow Can Artificial Intelligence use Big Data for Translating Documents?
33mDeep Learning at Booking.com
39mDeep Learning for Machine Translation: a paradigm shift
43mThe Evolution of Domain Transfer in the Era of Pre-trained Language Models
01h 38mTraining Large Language Models: Practices and Research Questions