A Branching Strategy For You?
summary
Claim: a photoreal talking avatar is generated from text using only six minutes of footage and a standard text to speech plus audio processing. Workflow uses short time Fourier + MFCC on 40 ms audio chunks, an autoencoder that keeps image data out of the latent code so audio drives motion, FFmpeg with optical flow and latent-code morphing for smooth frames, and Azure streaming and bot framework for delivery. Result is a deployable product for landing pages and chatbots that the demo shows working.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 2:31Project overview
- 5:08Photorealistic avatar use cases
- 7:17Data creation process
- 8:43Audio processing pipeline
- 10:27Audio features with MFCC
- 16:08Audio chunk sizing
- 19:16Autoencoder model architecture
- 22:13Multi-input encoder integration
- 24:43Face reconstruction inpainting
- 27:12Training and mask optimization
- 31:07Inference and frame generation
- 32:11Optical flow for transitions
- 33:37Frame interpolation methods
- 36:46Phoneme-based frame skipping
- 38:11Azure cloud deployment
- 39:58Q&A
related talks
19mFrom Demo to Deployment: Multimodal Conversational Agents
37mInventing Friends: ActionCable + AVS = 3
45mScalable Open AI GPT Applications in the Cloud
01h 04mShaving the Yak with AI - Devs want specs, Product Owners want speed
10mRapid Prototyping with AI
01h 24mBuilding your own software factory
58mWorkshop: Empowering Collaboration: AI Developer Experience - Your Bridge from Mod...
01h 31mAccelerate Business Delivery With AI and DevOps