Val Bercovici, WEKA & Daniel Kearney, Firmus | NVIDIA GTC '26
summary
Demonstrates a two-rack proof of concept that uses WEKA’s Augmented Memory Grid to arbitrage storage for KV cache and deliver 6.5× more tokens from the same GPUs and energy (equivalent to creating five and a half new data centers). Describes a heterogeneous stack where GPUs handle prefill, shared context memory holds large windows and LPUs handle fixed-latency decode, turning repeated prefills into an order-of-one operation and outlining Project Southgate’s planned 2.7 GW deployment.
AI-written summary · about
