CXL-ANNS: Software-Hardware Collaborative Memory Disaggregation and Computation...
summary
CXL-ANNS places billion-point graphs and embeddings in a CXL memory pool, adds a local DRAM cache of inner nodes (59% hit rate with 128 GB local memory) and offloads vector distance computation into custom logic on type‑3 CXL endpoints to cut data movement. A FPGA prototype plus cycle‑accurate simulator shows endpoint acceleration reduces transferred vector bytes by 21x and yields 111.1x higher QPS and 93.3% lower latency versus state‑of‑the‑art, and 3.8x throughput over an unlimited‑DRAM oracle.
AI-written summary · about
chapters (12)
- 0:00CXL-ANNS research intro
- 1:48Billion-scale ANN search challenges
- 3:33Graph-based ANN search
- 6:23CXL memory expansion basics
- 8:10CXL memory limits for ANN
- 9:36Two optimization observations
- 12:03CXL-ANNS design overview
- 13:29Graph traversal caching
- 15:34Distance calc acceleration
- 18:22Evaluation setup prototype
- 19:27Performance vs baselines
- 20:50Conclusion and future
related talks
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
13mHardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep...
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
43mHigh Speed Query Execution with Accelerators and C++
16mThe Clustering Strikes Back: Building Cost-Effective and High-Performance ANNS at Scale
01h 11mMachine Learning Acceleration in an Embedded World
27mComputers for Super-Human AI
16mCognitive SSD: A Deep Learning Engine for In-Storage Data Retrieval