On Dynamic Multimedia Pipelines for Opening Browsers to New Formats
summary
Builds a dynamic transcoding pipeline in WebAssembly that links a small main “solver” module to pluggable side modules (decoders/encoders) and runs inside the page. Integrations demonstrated: Web Components (universal canvas/image), MutationObserver fallback on decode failure, and an MP4-packed ServiceWorker that unpacks modules and feeds WebCodecs. Measured results: JPEG XL decode 17 ms native vs ~100 ms pipeline; video runs ~67 fps at 1080p; pipeline overhead comes from instantiation and linking.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:30Unsupported formats problem
- 2:37WebAssembly pipeline challenges
- 4:22Multimedia pipeline
- 6:31Web components approach
- 8:59Mutation observer decoding
- 10:44Service worker integration
- 12:56Performance benchmarks
- 14:20Conclusion
- 15:04Q&A: Instantiation time
- 16:32Q&A: Missing web features
- 17:35Q&A: WebAssembly features
related talks
21mOn the fly WebP decoding with Web Assembly and Service Workers (Main Stage)
12mExploring Video Editing in the Browser
30mOther browser APIs that are going to change video on the web
15mMediaPipe Web: Bringing cross-platform AI tech to the browser
24mVideo Editing in the Browser
27mService Workers + WebAssembly
19mPuppeteer + canvas + WebCodecs to replace
20mDecentralizing Video Streaming