How Intelligent Compute Accelerators (ICAs) are Shaping the Future of HPC & AI
summary
Links text and image spaces using CLIP 768‑d vectors trained on roughly 400M alt-tag pairs, and generates images by a denoising diffusion process implemented with a UNet that predicts removable noise rather than pixels. Conditioning uses classifier‑free guidance (run with empty prompt, scale prompt impact) to trade off fidelity versus prompt adherence. A sparse autoencoder visual steering pipeline makes clip embeddings monossemantic so interventions from example images reliably bias outputs (blue backgrounds, hats) and produces usable edits.
AI-written summary · about
chapters (12)
- 0:00Text to image models
- 5:35Neurons to neural networks
- 13:48CLIP linking text and images
- 21:56Denoising diffusion for images
- 28:01Building text to image models
- 34:50Interpreting models with autoencoders
- 38:02Visual steering applications
- 39:06Q&A seeds and reproducibility
- 40:50Sparse encoders for search
- 43:19Tools for interpretability
- 44:23Noise limits in diffusion
- 46:11Closing remarks
related talks
41mKEYNOTE: Image processing, AI, and autonomous systems
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
17mApproximate Caching for Efficiently Serving Text-to-Image Diffusion Models
45mScale Learning and Reasoning Across Heterogeneous Gradients and Semantics
22mEfficient Pretraining of Text-to-Image Models
01h 26mBuilding an Image Search Engine with Generative Computer Vision Models
18mDriving Innovation with AI
01h 00mData, visualization, and designing AI