To Cache or Not To Cache? a Tiered KVCache Storage System for Agent ...
Talkat KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 · 2026-09-07..09-09·
summary
Describes KVCache reuse for inference workloads (OpenClaw, Code Agents, chat sessions) and presents the Unified Cache Manager (UCM) lifecycle framework that adds eviction and tiering to KVCache storage. Covers workload analysis of data characteristics and retention patterns, predictive modeling of capacity and retention windows across models and hardware, and heuristic-driven retention/eviction policies that improve cache utilization and reduce inference cost.
AI-written summary · about
related talks
10mLightning Talk: KV-Cache Centric Inference: Building a State-Aware...
12mKV Cache and Long-Context Inference: Solutions for Inference at Scale
19mBeyond the Node: Scaling Inference with Cluster-Wide KVCache...
26mLMCache: Lower LLM Performance Costs in the Enterprise
31mGotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid...
17mCacheSlide: Unlocking Cross Position-Aware KV Cache Reuse for Accelerating LLM Serving
17mDroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants
30mDesigning Fast & Cost-Efficient Agents: The Power of Caching