TorchRec: A PyTorch domain library for large-scale recommender systems
summary
TorchRec is a PyTorch domain library that swaps nn.EmbeddingBagCollection for sharded, module-level equivalents, adds collectives and awaitable sharding strategies, and uses FBGEMM kernels for table-batched embeddings and optimizer fusion. Benchmarks show a typical training-loop speedup of 32x versus the baseline embedding wrapper on one GPU; using UVM caching keeps most embeddings on CPU and incurs ~2x slowdown (still ~16x faster than baseline) while a CPU-only embedding regime costs ~30% slowdown. Multi-GPU scaling is sublinear unless batch size is increased (4× batch size recovers near-linear throughput).
AI-written summary · about
chapters (11)
- 0:00Intro
- 0:28Agenda
- 1:40Model Parallelization
- 2:45Additional features
- 3:30Modeling changes
- 4:15Distributed environment
- 5:10Performance benchmarks
- 5:20Single GPU
- 5:55UVM Cache
- 6:40Batch Size
- 7:30Conclusion
related talks
22mHypeReca: Distributed Heterogeneous In-Memory Embedding Database for Training...
10mLightning Talk: Scaling Recommendation Systems To 2K GPUs and Beyond
34mAccelerating Sequential Recommender Models in the PyTorch Ecosystem
15mAccelerating Neural Recommendation Training with Embedding Scheduling
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
30mOur Journey With TorchTitan
24mTraining large scale models using PyTorch
38mMPI Meets Machine Learning: Unlocking PyTorch distributed for scaling AI workloads