HypeReca: Distributed Heterogeneous In-Memory Embedding Database for Training...
summary
HypeReca is a distributed in-memory key‑value embedding database that uses both GPU and CPU memory, implements decentralized indexing tables (DIT) with a local pipelined shard lookup and a ring-like schedule, and replicates a small R chunk of hot embeddings under a 2FP two-fold parallelism consistency scheme. On 32 GPUs over real datasets it reports 2.16–16.8× end-to-end speedup versus HugeCTR, TorchRec and TFDE, and the code is on GitHub.
AI-written summary · about
chapters (15)
- 0:00HyperReca system intro
- 1:01Deep learning rec models
- 2:46Comm overhead in training
- 4:10Embedding table access patterns
- 5:58HyperReca architecture overview
- 8:03Fine-grained data challenges
- 9:31Decentralized indexing tables
- 11:39Local DIT acceleration pipeline
- 13:29Ring scheduling to avoid conflicts
- 14:56Twofold parallelism strategy
- 16:01Optimizing R chunk size
- 17:28DIT throughput evaluation
- 19:17End-to-end training results
- 20:21Roofline model analysis
- 21:23Conclusion & Q&A
related talks
08mTorchRec: A PyTorch domain library for large-scale recommender systems
20mOPER: Optimality-Guided Embedding Table Parallelization for Large-scale...
15mAccelerating Neural Recommendation Training with Embedding Scheduling
45mBehind the hype: managing billion-scale embeddings in Elasticsearch and OpenSearch
21mESPN: Memory-Efficient Multi-Vector Information Retrieval
28mThe Memory Efficient Indexing for Vector Databases
49mDistributed Deep Learning
26mEfficiently serving LLMs at scale.