Pushing the Limits of In-Network Caching for Key-Value Stores
summary
OrbitCache caches whole key–value pairs by keeping packet recirculation loops in the programmable data plane instead of storing values in switch memory, enabling variable-length items and resolving NetCache’s size limits. Prototype runs on an Intel Tofino with RDMA clients and TommyDS servers, testbed of eight machines (4 clients, 4 emulated storage threads), preloading 128 cached items (NetCache used 10k small-item entries). Results show balanced service for highly skewed workloads, scales to 64 partitions, but recirculation ports become a bottleneck past ~128 cached packets and add microsecond latencies at low load.
AI-written summary · about
chapters (12)
- 0:00In-network caching intro
- 2:30NetCache fixed-size limits
- 5:26OrbitCache recirculation
- 7:13OrbitCache design challenges
- 8:17Request table and Q ops
- 10:27Cache replication serving
- 11:33Variable-length keys handling
- 12:40Implementation and testbed
- 13:43Performance vs NetCache
- 14:49Latency overhead analysis
- 15:54Trade-offs and cache limits
- 16:16Conclusion and future
related talks
23mFast Software Cache Design for Network Appliances
25mDistCache: Provable Load Balancing for Large-Scale Storage Systems with Distributed...
53mLeveraging Caching For Cloud Native Applications
19mFarReach: Write-back Caching in Programmable Switches
23mSize-aware Sharding For Improving Tail Latencies in In-memory Key-value Stores
28mAchieving 10Gbps Line-rate Key-value Stores with FPGAs
33mScaling KV Caches for LLMs: How LMCache + NIXL Handle Network and Storage...
13mBMC: Accelerating Memcached using Safe In-kernel Caching and Pre-stack Processing