Powering Your Generative AI Workloads with AMD and Open-Source ROCm
summary
AMD hardware plus open-source ROCm runs real generative workloads: PyTorch works on Linux, Optimum-AMD and Transformers run via Llama.cpp/AMA, and FlashAttention2, GPTQ and vLLM see partial support. A W7900 (48 GB) loads 70B LLaMA-family models; a 7900 XTX hits ~90% of an A100 on many models but drops to ~75% on the largest. Multi‑GPU needs a grub immu fix for DDP, torch.compile gives ~20% speedup, and a Mi210 with FlashAttention reduces fine‑tune time to under an hour on four cards. PyTorch support AMA Llama cpp demos
AI-written summary · about
chapters (10)
- 0:00AMD GPUs for generative AI
- 1:02What is ROCm
- 3:53AMD GPU hardware classes
- 5:20Software compatibility overview
- 9:12Tips for AMD GPUs
- 13:31Benchmarking vs Nvidia A100
- 19:07Live demo: local AI code assistant
- 22:59Running multiple models on GPU
- 27:27Fine-tuning with TorchTune
- 30:14Why choose AMD GPUs
related talks
03mKeynote: AMD & PyTorch: A Powerful Combination for Generative AI
05mBuild AI Anywhere with ROCm™ Software: AMD and PyTorch Bring...
13mFrom Silicon to Scale: Training at Scale
26mEfficient MoE Pre-training at Scale on AMD GPUs With TorchTitan
15mDeploy Hugging Face Models on Azure powered by AMD ROCm software
20mBringing PyTorch Monarch to AMD GPUs: Single-Controller Distributed Tra...
12mLightning Talk: PyTorch 2.0 on the ROCm Platform
25mOptimize productivity by meeting next-level demands of the AI era