The Colorful Truth of Automated Tests
summary
Claim: player APIs lie so tests assert the screen not the player. Built encoded test streams that place six red blocks every five frames and other big color blocks so a cheap webcam + Raspberry Pi can capture blurry output; simple image recognition reads frame position, per-bitrate unique color blocks and CDN-specific source content to tell bitrate CDN and ad asset. It runs on Smart TVs, avoids DRM issues, and exposes mismatches between reported and actual playback.
AI-written summary · about
chapters (10)
- 0:00Automated testing challenges
- 1:05Naive frontend automation limits
- 2:50Capture methods fail for video
- 3:31Webcams capture user views
- 4:36Simple test content design
- 5:40ABR quality and CDN testing
- 6:44Ad insertion with test streams
- 7:48Webcam test system in practice
- 8:11Live demo of webcam testing
- 9:13Conclusion
related talks
37mThe Colorful World of Visual Test Automation
34mTesting Live Video Streams At Scale
40mVisual regression testing: your strategy's blind spot
08mHow Big Buck Bunny helped track down bugs
39mYou're Testing WHAT?
43mTest automation on the living room devices
48mAudio and Video Testing Fundamentals
19mTesting things that don't want to be tested: A game streaming story