Future with Large Language Models
summary
Defines large language models as transformer-based nets since 2017 trained on petabyte-scale text with up to ~500 billion parameters running on GPUs and TPUs, powering Translate Search and Assistant and available on Google Cloud. Claims include emergent abilities that solve tasks not explicitly trained, concrete workflows of prompt tuning, heavier fine-tuning for domain tasks (legal summarization example) and distillation to compress teacher models into deployable student models. Notes massive compute and diverse web data as the practical enablers.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:18The Fourth Revolution
- 2:12Large Language Models
- 3:21Why now
- 4:56How things have evolved
- 6:37Language from large language models
- 7:45What are large language models
- 8:27Why are large language models different
- 9:50Foundation models
- 11:03Use Foundation models in production
- 14:41Conclusion
related talks
27mLarge Language Models and the Future of AI
30mLarge Language Models and Prompt Engineering for Developers
43mBeyond the Hype: A Realistic Look at Large Language Models
51mLarge Language Models: Friend, Foe, or Otherwise
40mThe Future of Language Models
19mSystem for Scaling Large Language Models
51mTowards a Pragmatic Understanding of Large Language Models
44mBuilding projects with language models