Building models for TPU devices
summary
Shows how to train a TensorFlow model, apply 8-bit quantization, freeze and convert it to TensorFlow Lite, then compile for the Edge TPU (Coral) and deploy to small devices including Intel accelerators and Jetson-like boxes. Presents measured times (model load ~30s, compile ~1,000s), notes quantization reduces model size with single-digit percent accuracy loss, and explains constraints such as constant tensor shapes/parameters and unsupported ops falling back to the host CPU (causing partial offload and occasional false positives in object detection). Discusses Edge TPU performance claims (~4 TOPS) and benefits for latency, cost, and privacy.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:13Computing evolution
- 11:03What is machine learning?
- 18:45What is a TPU and its architecture
- 24:30Edge TPU devices
- 28:04Demo: Model deployment on TPU
- 35:10Building models for TPU
- 47:16Edge computing benefits
- 50:26TexOS 2024 announcement
- 51:30Q&A
related talks
14mOptimize your TensorFlow Lite models
21mThings I learned running neural networks on microcontrollers
34mTensorFlow Lite
20mPyTorch Edge: Developer Journey for Deploying AI Models Onto Edge Devices
39mMachine Learning on Your Device: The Options
36mHigh-Performance Distributed Tensorflow
37mDeep Learning Neural Network Acceleration at the Edge
43mTensorFlow for Mobile Developers