Revisiting Secondary Indexing in LSM-based Storage Systems with Persistent Memory
summary
Claims a complete design called PERSEID that uses a PS3 two-layer PM index, a hybrid PM-DRAM hash validation to avoid expensive primary-table reads, and two primary-table search optimizations: sequence-number filter and worker-active parallel search. Implementation runs on Intel Optane DC persistent memory with 100M primary keys and 4M secondary keys and 1KB records. Evaluation shows PERSEID outperforms existing PM indexes by 3–7× and yields about two orders of magnitude improvement over prior LSM-based secondary-indexing techniques, demonstrating the approach works.
AI-written summary · about
chapters (10)
- 0:00Secondary indexing in LSM trees
- 3:57Challenges in LSM-based systems
- 7:06PM-based indexing limitations
- 9:55PERSEID’s hybrid hash table
- 12:23Optimizing index queries
- 15:34Evaluation setup
- 16:37Index-only query results
- 17:42Multi-threaded performance
- 18:47Non-index query optimizations
- 19:10Conclusion
related talks
14mSEPH: Scalable, Efficient, and Predictable Hashing on Persistent Memory
17mPreview: Persistent Memory
16mAerospike + Intel® persistent memory
35mDeveloping Software for Persistent Memory
17mPIMLex: A High-Performance Learned Index with Processing-in-Memory
35mDeveloping Software for Persistent Memory
43mPromise and Pitfalls of Persistent Memory
29mSLM-DB: Single-Level Key-Value Store with Persistent Memory