GRACE: Loss-Resilient Real-Time Video through Neural Codecs
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
A neural video codec that jointly trains encoder and decoder under simulated packet losses and randomly zeroed tensor partitions yields graceful quality degradation: comparable to H.265 at zero loss, yet maintaining higher quality as loss increases. Training uses a random 0/1 mask on coded tensors to mimic packet drops and partition loss during inference. Evaluations show 95% fewer undecodable frames and 90% shorter stall duration, plus a 38% higher MOS over baselines, running in real time on GPU and mobile.
AI-written summary · about
chapters (8)
- 0:00Introduction to video loss
- 1:47Existing solution analysis
- 3:01Grace framework design
- 5:16Grace technical details
- 8:19Experimental evaluation
- 11:10Performance benchmarks
- 12:36Real-world testing
- 14:30Conclusion and results
related talks
16mSwift: Adaptive Video Streaming with Layered Neural Codecs
17mContent Aware Encoding for low latency live streaming encoders using deep learning
22mAI Enhanced GPU Video Coding: Achieving Joint High Compression Efficiency and Throughput
15mFast Lossless Depth Image Compression
17mMorphe: High-Fidelity Generative Video Streaming with Vision Foundation Model
17mYour Player is Smarter Than You Think: Self-Learning in Adaptive Streaming
18mOn-device Video Playback Upsampling
21mWhat to do after per-title encoding