Why People Really Abandon Playbacks

on YouTube ↗46m

" summary

Many observational studies misidentify cause and effect; YouTube A/B experiments and ABR analysis show faster playbacks actually lower retention and paradoxes like fatal errors drive more usage because dedicated users work around failures. False 400–600σ results come from slicing on dependent playback events; true evaluation uses an independent daily session ID and population-level metrics. Concrete numbers: 2.5% interact with quality daily, 25% of watch time comes from those users, 10,000 users tap-to-retry 100+ times per day, and hundreds of thousands spend more time buffering than playing.

AI-written summary · about

" chapters (22)

1. 0:00Intro
2. 1:12Rebufferings incur abandonment rates six times higher than start-up latency'
3. 1:37They're wrong
4. 3:35Queasy-Experimental Design
5. 3:54Please save questions for the end
6. 6:23Rare events are associated with longer sessions
7. 11:28Slice the population using an independent hash
8. 11:48Sum up metrics across population slices
9. 12:18Do riskier analysis to figure out what experiment to run
10. 12:53Wrong metrics
11. 18:31Pick a few simple metrics that mean the most to your business
12. 18:50User happiness!!!!!
13. 24:02Your users use your service
14. 25:15Three sips every time the video was 1080p but the audio was 48kbit
15. 29:18The most error-prone popular device in North America
16. 31:16Can't measure the impact of a fix
17. 32:20Restoring to six-month-old levels causes a hit
18. 33:24Improve your fundamentals. You'll find out why in a year.
19. 33:34Your service is growing
20. 35:52Self-sabotage
21. 36:30Using overlapping TCP connections to grow throughput
22. 37:21Our service isn't YouTube

" related talks

• 34mIdentifying Root Cause via Playback MetricsMatt Fisher · Demuxed 2018
• 14mAnalyzing Video Metrics like Richard FeynmanBen Dodson · Demuxed 2019
• 13mUltra-High Resolution Playback Visualization and its ApplicationsSteve Robertson · Demuxed 2017
• 17mMetrics Driven Obsession with Viewer ExperienceKhawaja Shams · Video @Scale 2024
• 17mPredict, Prune, Play: Efficient Video Playback Optimization Under Device...Mohammad Alizadeh, Pouya Hamadanian, Arash Nasr-Esfahany, Harsha Sharma, Zahaib Akhtar · NSDI '26
• 33mThe Challenges of Testing Video Playback at ScaleIan Goddard · TestCon Europe 2019
• 22mProviding better video experiences for the next billion users.Denise Noyes · Demuxed 2020
• 16mImproving Streaming Experience with Bayesian Optimization, from AB to...Guillaume du Pontavice · Demuxed 2021