Seeing Through the Lens of Transformers: A New Era in Computer Vision
summary
Vision Transformers replace convolutions by splitting images into patch-based tokens, flattening and linearly projecting them, adding position embeddings, then using multihead self-attention to learn global relationships. Google trains ViT-like models on about 300 million images (nonpublic) and shows higher accuracy than ResNets at scale, but ViTs are data‑hungry, parallelizable, and weaker at fine-grained segmentation; practitioners usually download pretrained ViT checkpoints, fine‑tune for classification, or use hybrid CNN+Transformer designs to recover localization.
AI-written summary · about
chapters (12)
- 0:00Vision Transformers intro
- 1:12Why ViTs gained traction
- 3:39Classical convolutional neural networks recap
- 6:06From NLP Transformers to Vision Transformers
- 9:18ViT architecture
- 11:51Global relationships in ViTs
- 15:31Data requirements and fine-tuning Vision Transformers
- 18:25Attention maps in ViTs
- 20:14Performance comparison with CNNs
- 22:05ViT limits for segmentation
- 23:29ViT evolution summary
- 24:36Q&A
related talks
43mVisual Attention is All You Need: Introduction to Visual Transformers
41mBridging Pixels and Words: The Transformer Vision Revolution
17mHow Transformers Finally Ate Vision
14mA Primer to Modern Day Computer Vision
35mUnpacking Scattering Vision Transformer: A New Dawn in Computer Vision
17mVision AI in 2025
28mLarge Multimodal Models: Understanding the State of the Art in Compute
42mDeep Learning for Computer Vision