SAM 2: Segment Anything in Images & Video
summary
Segment Anything Model 2 is a transformer with streaming memory that performs promptable segmentation on images and videos, trained on a new SAV dataset with ~650k masks across ~50k in-the-wild videos to become the largest video segmentation set to date. It stores spatial memories (about six frames) plus object pointers and an occlusion score, runs a hierarchical ViT at 1024 px, and claims 6x faster on images and 3x fewer interactions for video while improving accuracy.
AI-written summary · about
chapters (11)
- 0:00SAM 2 introduction and goals
- 0:48SAM 1 recap and capabilities
- 3:57Video segmentation challenges
- 7:07SAM 2 model and SA-V dataset
- 9:14SAM 2 architecture with memory
- 11:00Decoder updates for video
- 13:05Largest video segmentation dataset
- 15:13Qualitative results and performance
- 16:59Quantitative evaluations
- 18:49SAM 2.1 limitations and improvements
- 19:32Launch, use cases, and developer suite
related talks
32mEverything you wanted to know about VLMs but were afraid to ask
15mTinySeg: Model Optimizing Framework for Image Segmentation on Tiny Embedded Systems
33mArchitecting Scalable Multi-Modal Video Search
35mLarge-Scale Video Intelligence
01h 29mHuman-like Visual Search Application with Small Data
45mTech Tutorial: Image segmentation
40mSmart Crop and Smart Preview via Video Understanding
46mBuilding computer vision applications Q&A