Deepdive into Gemma 3
summary
Gemma 3 ships as four open checkpoints (1B, 4B, 12B, 27B) with pre/post-train weights and quantizations bfloat16, float8, 4, and q4_0; the 1B in q4_0 fits under 700M memory and the 27B Gemma 3-27B ranks top 10 on LMSYS while outperforming models 15× its size. Models train with a frozen SigLIP vision encoder and support interleaved image+text, OCR and pan-and-scan crops, offer 128k context length via RoPE rescale ×8 and interleaved local/global attention to cut KV cache, and expose function calling for programmatic control. q4_0 <700MB footprint
AI-written summary · about
chapters (8)
- 0:00Gemma 3 launch
- 1:47Model sizes and capabilities
- 4:37Architecture and pre-training
- 8:53Post-training and benchmarks
- 13:53Native multimodal
- 18:36Multimodal demos
- 22:17Multilingual advancements
- 26:00Closing and future
related talks
19mGemma 4 Deep Dive
10mDemo: Using Gemma with the Hugging Face ecosystem
15mGemma, DeepMind's Family of Open Models
15mRunning Google's Gemma LLMs in the browser with MediaPipe Web
16mOpen models in the Gemini era
39mFlutter Project with Gemma: A State-of-the-Art Open Model
10mAI for everyone with Gemma
40mBuilding amazing Front-End Project with Gemma: A State-of-the-Art Open Model