Understanding and Tackling the Hidden Memory Latency for Edge-based Heterogeneous
summary
Shows that unified memory on Jetson iGPU platforms initializes most data on the CPU, making initialization account for up to 90% of overall latency. Builds an eUMM prototype that moves initialization into GPU memory, implements prefetch and CUDA streams to overlap page mapping and init, and evaluates on Jetson TX2 and Xavier AGX. For large matrices GPU-side init cuts initialization and kernel mapping latency and speeds end-to-end execution; for very small buffers GPU kernel launch overhead can outweigh gains.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:34Unified Memory Model
- 2:43Hidden Memory Latency
- 6:20Enhanced Unified Memory Management
- 7:10GPU initialization
- 7:51Benchmarks
- 10:05Conclusion
related talks
14mFreedom from memory interference
26mNeuOS: A Latency-Predictable Multi-Dimensional Optimization Framework for DNN-drive
38mDynamic Data Structures and Memory Management on GPUs
52mKeynote: Domain-Specific Memory Management?
46mBuilding Mixed Criticality Linux Systems with the Jailhouse Hypervisor
20mGray-in-Young: A Generational Garbage Collection for Processing-in-Memory
22mPreview: Hardware and Offloads
25mBridging the Memory Hotness Gap in Edge Systems with Hotness-Segregated Object Allocation