AI'll Be Back: Exploring Diffusion Models for Image, Audio & Video
summary
Live demos show images from Midjourney, music from SunoAI, and animations driven by RunwayML plus a hands‑on Open Sora video reimplementation that runs with only ten diffusion steps so results look rough. The talk explains Latent Diffusion, CLIP guidance, spectrogram‑to‑audio diffusion, and video patching/latent tricks, gives version history (Midjourney v1→v6) and concrete failure modes, and calls out legal and data provenance concerns.
AI-written summary · about
chapters (28)
- 0:00Generative AI in media
- 1:17AI hype to real-world apps
- 2:39Understanding large language models basics
- 4:45Tokenization and embeddings
- 6:56Semantic meaning in vectors
- 8:42Attention mechanism
- 10:48LLM components
- 12:33Text-to-image evolution
- 14:40Diffusion models and refinement
- 15:43Noise to denoising
- 17:15Markov chains in diffusion
- 19:25Image generation with diffusion
- 21:13Live image demo
- 22:40Guidance in image restoration
- 24:24Autoencoders and latent diffusion
- 26:30Text-to-audio with spectrograms
- 28:18Audio diffusion models
- 29:24Text-to-video challenges
- 31:14Temporal coherence in video
- 33:20Latent diffusion for video
- 34:25Open Sora video demo
- 36:51Video diffusion transformers
- 39:00AI physics and realism gaps
- 40:33Ethical AI training data
- 43:25AI impact on labor
- 44:29Disinformation and rights
- 45:53Text-to-video progress
- 46:18Q&A
related talks
01h 27mImage Generation with AI
45mGenerative AI for Image and Video Production
10mDiffusion models
20mRealtime Video AI with Diffusion Models
45mDiffusion to Prod: vLLM-Omni Unifies Multimodal Inference
10mLightning Talk: Diffusers: Bringing Cutting-Edge Diffusion Models to the Masses
23mDiffusion-Based Sound Synthesis in Music Production
39mDemocratizing Diffusion Models with Diffusers