Size-aware Sharding For Improving Tail Latencies in In-memory Key-value Stores
summary
Minos implements size-aware sharding in an in-memory KV store (DPDK, 8 cores, 40 Gb NIC) by partitioning cores into disjoint small/large sets, using a runtime CDF threshold (e.g., 1.5 KB) and packet-count load estimates to auto-allocate cores. Small requests use hardware RXQ dispatch so only rare large ops incur software handoff. Results show 99th percentile up to two orders of magnitude lower and throughput up to 7.4× higher versus competitors for workloads with <1% large items, with a noted 2× throughput cost near saturation.
AI-written summary · about
chapters (14)
- 0:00Start
- 0:16Contributions
- 0:43Tail latencies in high fan-out applications
- 6:15Implementation
- 6:52Minos design challenges
- 7:14Main insight
- 9:27Dynamic core allocation
- 10:31Request size unknown a priori RX queues
- 14:05Throughput vs overall 99th latency
- 14:22Minos vs early binding
- 15:07Minos vs stealing
- 15:48Minos vs late binding
- 16:26Higher throughput
- 16:44Trade-off: 99th latency of large operations
related talks
17mPushing the Limits of In-Network Caching for Key-Value Stores
18mSplitKV: Splitting IO Paths for Different Sized Key-Value Items with Advanced
13mDifferentiated Key-Value Storage Management for Balanced I/O Performance
16mOneSidedMW: Managing Disaggregated Memory Efficiently, Flexibly, and Securely with RNIC
19mFineMem: Breaking the Allocation Overhead vs. Memory Waste Dilemma in Fine-Grained...
15mMitigating Resource Usage Dependency in Sorting-based KV Stores on Hybrid...
15mImproving Performance of Flash Based Key-Value Stores Using Storage Class Memory...
26mLeveraging Moore's Law to Optimize Database Performance