HotRAP: Hot Record Retention and Promotion for LSM-trees with Tiered Storage
summary
HotRAP is a RocksDB-based key-value store that implements record-level promotion using an additional small on-disk LSM called route to track hotness with exponential smoothing and Bloom-filter checks (no extra reads). Two promotion pathways move hot records into fast storage: promotion-by-compaction (scan compaction ranges and retain hot keys) and promotion-by-flush (mutable promotion buffer flushes safe-to-promote records). Experiments show up to 1.6× throughput over the second-best tiered LSM on skewed YCSB and up to 1.5× on Twitter traces, with much lower memory use.
AI-written summary · about
chapters (9)
- 0:00HotRAP and LSM-trees intro
- 1:44Hot records in slow storage
- 3:14Existing retention limits
- 4:39HotRAP overview and paths
- 6:05Promotion by flush and checks
- 8:11Hotness tracker design
- 9:39Experimental results
- 10:39HotRAP advantages
- 11:00Q&A
related talks
16mHolistic and Automated Task Scheduling for Distributed LSM-tree-based Storage
14mSMART: A High-Performance Adaptive Radix Tree for Disaggregated Memory
35mSystems @Scale Summer Q&A Week 3
19mBuilding a Performant Configuration-Store Layer
39mHow we isolate streaming ingest from search using RocksDB
16mFrom Sampling To Full Visibility: Scaling Tracing To Trillions of Spans
15mTreebeard: A Scalable and Fault Tolerant ORAM Datastore
15mROLEX: A Scalable RDMA-oriented Learned Key-Value Store for Disaggregated Memory Systems