De-mystifying PyTorch for ASICs: When (and Why) To Move Your Development To AI A...
summary
Benchmarks compare PyTorch/XLA (TPU) and TorchNeuron (Trainium) with side‑by‑side code diffs and real runs: fine‑tuning Llama 4, Gemma 3, Qwen 3 plus ResNet50, ViT and CNNs. Results show 8× H1 TPUs finish ResNet50 in about 2 hours and come close to H100s after the long compiler tax initial compilation; repeated runs reuse lazy tensors so throughput spikes. Common failures include simple out of memory (32 GB TPU vs 33 GB allocation solved by halving sequence length) and hanging compile steps; migration needs modest code edits but watch XLA semantics.
AI-written summary · about
chapters (10)
- 0:00Introduction and setup
- 1:50ASICs and frameworks overview
- 3:55GPU vs TPU architecture
- 6:24PyTorch XLA and lazy tensors
- 8:30Performance benchmarks comparison
- 9:37Understanding compiler tax
- 11:27Common ASIC errors
- 12:31Migration challenges
- 13:15Future updates and TPUs
- 14:18Accessing free TPUs
related talks
17mEnable Generative AI Everywhere with Ubiquitous Hardware and Open Software
23mHow Pytorch Powers Training Inference
26mEverything Everywhere all at Once: vLLM...
16mtorchtune: Easy and Accessible Finetuning in Native PyTorch
26mTraining Large Deep Learning Models: PyTorch vs JAX
32mKeynote: AI & the stuff built for AI - are they actually useful for data science?
22mGenAI Training In Production: Software, Hardware & Network Considerations
12mDemo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM