Aya: Expanding the Worlds AI serves
summary
A research lab builds YA: a multilingual dataset of 513M datapoints across 101 languages with 3,000 collaborators, 56 language ambassadors and 204,000 human annotations, then trains and publishes models (a 32B and 8B model plus a demo on WhatsApp). The pipeline uses instruction tuning, translation augmentation, template generation, rigorous data pruning (they removed 50% English and 44% non‑English), cross‑lingual transfer and human eval; restricting capacity to 23 languages yields state-of-the-art on those languages, and the 8B model runs on a single GPU.
AI-written summary · about
chapters (9)
- 0:00Introduction to multilingual AI
- 1:02The challenge of language coverage
- 3:54Obstacles in multilingual modeling
- 7:12Data inequality and its impact
- 10:52Breakthroughs in generative AI
- 14:52Training techniques and cross-lingual transfer
- 19:32The Aya project
- 21:59Improving data quality and evaluation
- 25:33Discussion and concluding insights
related talks
44mKeynote: The Art of Language: Mastering Multilingual Challenges in LLMs
44mKeynote: The Art of Language: Mastering Multilingual Challenges in LLMs
14mAI for a Billion Voices: Taking AI where there is Maximum Impact
19mTraining Data and its Impact on Cultural Understanding & Ethical Values in Giant AI Models
51mAI for every language
13mCustom Training of AI for Language Services: Translation is Just the Beginning
28mAI’s Next Great Challenge: How to create AI that truly understands us?
43mBreaking the language barrier: how do we quickly add multilanguage support in our AI application?