Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
Caches and reuses intermediate diffusion noise states to skip early sampling steps, claiming 21% GPU compute savings, 19.8% end-to-end latency reduction, and ~19% dollar savings on real workloads. CLIP embeddings map prompt similarity to a skip K (tested on SDXL with 50 steps, sometimes skipping 20 steps for ~40% compute saved), retrieval via a vector DB, storage on EFS, and an LC-BFU eviction that scores frequency × skip savings. A one-class SVM match predictor avoids unnecessary nearest-neighbor queries; FID/CLIP metrics and a user study (≈79% acceptance) show near-vanilla quality.
AI-written summary · about
chapters (6)
- 0:00Context and motivation
- 1:21Diffusion model challenges
- 3:31Proposed caching approach
- 10:04System architecture
- 12:16Performance validation
- 13:18Conclusion and results
related talks
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics
17mDroidSpeak: KV Cache Sharing Across Fine-tuned Model Variants
22mEfficient Pretraining of Text-to-Image Models
47mHow Intelligent Compute Accelerators (ICAs) are Shaping the Future of HPC & AI
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
30mHow to train a large diffusion model
01h 16mOptimizing workloads with Transformers and Diffusers