Foundation Models: Blueprints for success
summary
Describes building and running foundation models using Optimum Neuron, the AWS Neuron SDK and Trainium/Inferentia on SageMaker, with concrete tradeoffs: compiling LLMs can take up to 60 minutes unless using precompiled Hub artifacts, Trn1n32xl offers 1.6 Tb network for multi-node training, and Inferentia claims ~90% higher throughput per dollar for diffusion models. Shows working Spaces running Llama 7B and Stable Diffusion, a Neuron trainer that enables tensor and pipeline parallelism across devices, and a retrieval stack using embeddings + OpenSearch.
AI-written summary · about
chapters (7)
- 0:00Introduction to foundation models
- 3:16Purpose-built hardware overview
- 7:38Hugging Face and AWS partnership
- 12:40Optimum Neuron library demo
- 19:05Training models on Trainium
- 21:34Choosing the right model
- 24:04Reference architecture for RAG
related talks
42mFine-tune and deploy LLMs from Hugging Face (AIM304)
25mSponsored Session: Accelerating GenAI Inference: From AWS Deep Learning...
20mFoundation Models for Big Data and Prompt Tuning
31mHow Amazon built Nova FMs using Amazon SageMaker
54mDeep learning on AWS with NVIDIA: From training to deployment
28mComputing Large Foundational Models
35mLLM Customization
02mInferentia2