What's New for Dynamic Shapes in PyTorch 2.1
summary
PyTorch 2.1 moves dynamic shapes to Beta: dynamic detection is now automatic (first compile specializes, later runs upgrade changing dims), and training with variable batch sizes works; logging and export report which dims become dynamic and why. New APIs let users force MarkDynamic/MarkStatic on tensor dims, and there is cograph recordings support that reuses memory across recordings. Data-dependent ops like item and nonzero are still mainly on nightly. Performance is slightly worse than static; warm up on representative sizes to avoid suboptimal kernels.
AI-written summary · about
chapters (10)
- 0:00Dynamic shapes in PyTorch 2.1
- 2:08Dynamic shapes Beta in PyTorch 2.1
- 3:32torch.compile for dynamic shapes
- 7:47MarkDynamic and MarkStatic
- 8:08Cograph support
- 9:10Dynamic shapes performance
- 12:01Dynamic shapes in torch.export
- 14:29Data-dependent operations
- 19:232024 roadmap
- 20:48Q&A
