When All Context Matters: Extended Cache Augmented Generation
summary
Claims a new method called Extended Cache Augmented Generation that shards rapidly changing, fully relevant document collections across multiple parallel CAGs, lets a supervisory model interrogate buckets and synthesize answers, and shows faster rebuilds than recomputing GraphRAG while improving accuracy over simple RAG. Implements per-bucket KV caches, balances documents randomly to minimize bucket count, and explores KV cache cost vs lifetime optimizations; demonstrates practical trade-offs and that the system works at lower latency.
AI-written summary · about
related talks
39mRAGs to Riches: How Our Content Affects Retrieval Augmented Generation
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production
50mRetrieval Augmented Generation Beyond Toy Demos
14mSmarter RAG, Smaller Bill: Optimize for Performance and Price
49mContext is King
51mDeveloping a Custom RAG System
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
42mTo RAG or not to RAG