Reconstructing 3D from Compressed Video: An AV1-Based SfM Pipeline

on YouTube ↗08m

" summary

Repurposes AV1 motion vectors as correspondences so SfM skips costly keypoint detection and matching, implemented as research Python code that extracts block vectors from bitstreams, prunes bad vectors by cosine similarity, and feeds matches into an incremental reconstruction pipeline. Claims median 42% front-end time reduction, up to 8× more 3D points with a median reprojection error of 0.5 px, and demonstrates dense usable point clouds from consumer video that materially speed reconstruction.

AI-written summary · about

" related talks

• 18mCoVA: Exploiting Compressed-Domain Analysis to Accelerate Video AnalyticsJongse Park, Jinwoo Hwang, Minsu Kim, Daeun Kim, Seungho Nam, Yoonsung Kim, Dohee Kim, Hardik Sharma, Minsu Kim, Daeun Kim, Yoonsung Kim, Dohee Kim · USENIX ATC '22
• 33mArchitecting Scalable Multi-Modal Video SearchSebastiano Milardo, Pietro Piccini · PyData Paris 2025
• 17mGemino: Practical and Robust Neural Compression for Video ConferencingMohammad Alizadeh, Pantea Karimi, Vibhaalakshmi Sivaraman, Vedantha Venkatapathy, Mehrdad Khani, Sadjad Fouladi, Frédo Durand, Vivienne Sze · 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24)
• 12mMeasuring live video quality with minimal complexity, now available for everyone!Jan De Cock · Demuxed 2024
• 17mYuZu: Neural-Enhanced Volumetric Video StreamingAnlan Zhang, Chendong Wang, Bo Han, Feng Qian · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 25mOptimal Video Compression Using Pixel Shift TrackingHitesh Saai Mananchery · Conf42 Machine Learning 2025
• 21mWhat to do after per-title encodingBen Dodson, Nick Chadwick · Demuxed 2018
• 24mBeyond the Frame: Using Computer Vision to Unlock the Potential of Video DataPaula Oses · RTC.On 2023