Building a Professional Video Editor on the Cloud, Powered by Machine Learning
summary
Runway converts every ML output into HLS model streams so the browser can request processed segments on demand; interactive tools use a click-to-keyframe model plus a propagation network with space‑time attention and edge refinement that predicts soft alpha for compositing. Training uses a probabilistic synthetic click simulator to match real user correction patterns. Latency tradeoffs include smaller backbones and quantization; pipeline supports random access and parallel prefetching, and the super slow motion feature upsamples 30 FPS to 300 FPS (10x) in production.
AI-written summary · about
chapters (8)
- 0:00Runway and ML video editing
- 2:55Runway workflow simplification
- 6:45Cloud-based ML video processing
- 10:22Latency and stability challenges
- 11:45Human-in-the-loop refinement
- 13:10Green Screen rotoscoping
- 17:42Random access and prefetching
- 19:06Future text-based editing
related talks
24mVideo Editing in the Browser
43mHigh performance video editing for the mobile web
15mUsing The Power of Edge Compute To Enhance Streaming Delivery
25mBetting on the web: Ensuring fine-tuned performance for dynamic web apps
13mVideoCompositor: a Case Study of Live Video Composing
12mExploring Video Editing in the Browser
16mInterview
26mBandwidth Ballet: Optimizing Video Performance for Web Maestros