Direct Access, High-Performance Memory Disaggregation with DirectCXL
summary
Built a full CXL v2 prototype—DirectCXL prototype—including a CXL root-port CPU, CXL endpoint memory device, reconfigurable switch and a Linux runtime that exposes remote DRAM via CXL.mem load/store access and a Linux character device + mmap path. Measured microbenchmarks show single-cache-line loads up to ~8× faster than RDMA (and ~24× when avoiding RDMA software stacks). End-to-end on a DLRM embedding workload is 2.8× faster for inference and 33.3× faster overall versus an InfiniBand RDMA setup.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:29High-Level Summary
- 1:57CXL: Computing Express Link
- 6:01Prototype: CXL Compute Blade & CXL Memory Blade
- 6:49Prototype: CXL Switch
- 7:35DirectCXL's Software Runtime
- 8:39Demo #1: RDMA vs. DirectCXL
- 9:35RDMA initialization
- 9:58Performance comparison with RDMA
- 10:33Demo #2: Memory Disaggregation
related talks
42mMessing up Your NUMA Topology with CXL
11mXerxes: Extensive Exploration of Scalable Hardware Systems with CXL-Based Simulation...
17mDuhu: Shared Disaggregated Memory for Distributed Data Processing Frameworks
18mRDMA is Turing complete, we just did not know it yet!
45mBeyond Machine Boundaries: Experience with Memory Disaggregation
19mFast Distributed Transactions for RDMA-based Disaggregated Memory
17mCylon: Fast and Accurate Full-System Emulation of CXL-SSDs
23mTeRM: Extending RDMA-Attached Memory with SSD