Live demo: Gemma 3 on Jetson Orin Nano
summary
Jetson Orin Nano boots Jetpack, flashes via SDK Manager, and runs Gemma 3 (demo uses a Gemma 3 4B build served with Ollama) after increasing swap to 16 GB; live image Q&A and timetable text extraction work (model reads “3:35”) and throughput measures about 14–15 tokens/sec. Hardware specs cited: 67 TOPS and 102 GB/s memory bandwidth at 25 W. Also demos of voice via RAC, overnight multiagent chat using llama.cpp agents, and 12B models run extremely slowly but do complete.
AI-written summary · about
chapters (15)
- 0:00Jetson Orin Nano intro
- 1:29Hardware upgrades and specs
- 3:09Jetson AI Lab resources
- 3:52Unboxing and setup
- 5:22Installing Jetpack
- 6:27Live demo setup
- 7:31Running Gemma 3 4B
- 9:00Image recognition demo
- 10:49Text extraction demo
- 11:53Describing cat images
- 13:03Performance metrics
- 14:07Voice interaction demo
- 15:12AI agents example
- 16:14Testing 12B model
- 17:25Call to action
related talks
08mDemo: Gemma on-device with MediaPipe and TensorFlow Lite
08mDemo: JAX, Flax and Gemma
24mAccelerating AI on Edge
16mGemma 4: Compacting Intelligence for the Edge
41mBuild intelligent Android apps with Google's AI
12mDemo: Rapid prototyping with Gemma and Llama.cpp
46mSmall AI is the next Big Thing
21mFrom 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents