Andy Pernsteiner, Vast | NVIDIA GTC '26
summary
Claims a 10X improvement in inference from a single GPU server by offloading KV cache attention data from HBM to storage using NVIDIA’s Dynamo inference engine and tuned KV caching, built on GPUDirect Storage and NFS over RDMA. Describes open-source pipelines based on NVIDIA blueprints for document and video ingestion, cites tripling exabyte deployments year over year, an integrated policy model for security and multi-tenancy, and a global namespace plus data engine that orchestrates compute near data.
AI-written summary · about
