Millions of hours: finding duplicates in VK Video

on YouTube ↗47m

" summary

VK Video receives hundreds of millions of new videos and needs to detect copies to protect creators. The talk explains the system built to find duplicate or copied videos at large scale, covering the engineering challenges, how detection works under heavy load, and measures taken to minimize costly false positives.

AI-written summary · about

" chapters (15)

1. 0:00Introduction
2. 1:10Finding video duplicates
3. 2:35System load and requirements
4. 5:06Video decoding: CPU vs GPU
5. 10:54Frame fingerprinting
6. 15:54Video fingerprint aggregation
7. 18:25Storing fingerprints in S3
8. 19:09Faiss vector search
9. 21:20Vector quantization
10. 25:54Vector storage and indexing
11. 28:41Solution scalability
12. 31:28System fault tolerance
13. 33:26Processing historical content
14. 34:09Summary and performance
15. 35:13Q&A

" related talks

• 35mLarge-Scale Video IntelligenceIrene Donato, Antonino Ingargiola · PyData Amsterdam 2025
• 37mHow we generate 130,000 shorts per month using AIAleksandr Sokolov · Saint HighLoad++ 2025
• 19mThe Evolution of Culture and Entertainment in a Digital WorldDavid Kenny, Ole Obermann · DLD Munich 24
• 49mDigital Storytelling: Surfing the Video ExplosionShahrzad Rafati, Stefan Schulz, Henry Blodget, Jimmy Maymann, Jamal Edwards · DLD 2014
• 29mKnowledge Graph–Driven Feedback Loops to Optimize TikTok Shop Video MarketingTaeyang Kim, David Fagerburg · NODES 2025
• 30mWhich episode? Using AI to streamline media content operationsAnushri Mainthia, Sally Hubbard · AWS re:Invent 2020
• 30mVisual Storytelling in the World of Algorithmic NarrativesMel McVeigh · Interact London 2019
• 11mOnline Video Challenges, Trends and SolutionsRob Glaser · DLD 2014