Real-time video upscaling using CNNs and WebGPU
summary
Uses WebGPU compute shaders to run a six‑layer CNN that does about 3,000 MACs per pixel on incoming video frames, importing video frames zero‑copy when possible and falling back to pass‑through on missed deadlines. Targets 1080p/1440p at 60 fps (≈125M pixels/sec), reporting ~8–9 ms per frame on WebGPU, 7.2 ms when compiled from PyTorch to CoreML and ~5 ms on an NPU. Implements FP16 quantization, loop unrolling, and weight blobs loaded as base64.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:03Real-time streaming challenges
- 2:27Super resolution overview
- 3:53Super resolution pipeline
- 4:54Video frames in WebGPU
- 5:59WebGPU compute shaders
- 7:51CNN with WebGPU shaders
- 9:15Neural network integration
- 11:04Performance benchmarks
- 12:09Key lessons
related talks
30mBeyond the Web of Today
17mRunning Intelligence at the Edge: Web AI in Action
24mVideo Editing in the Browser
17mMorphe: High-Fidelity Generative Video Streaming with Vision Foundation Model
38mWebGPU: Next-generation 3D Graphics on the Web
22mAI Enhanced GPU Video Coding: Achieving Joint High Compression Efficiency and Throughput
34mWhat WebGPU means for Web platform?
24mSuper Resolution: The scaler of tomorrow, here today!