High-Performance Distributed Tensorflow

on YouTube ↗36m

" summary

Demos show concrete optimizations: 8-bit quantization and weight rounding shrink a 32k float model to ~13k and activation calibration is needed; batch-normalization folding bakes means/variances into weights to remove runtime multiplies; TFCompile plus Excel A compilation can strip a TensorFlow runtime from ~30 MB to ~600 KB for embedded inference; Graph Transform Tool runs post-training passes. Load tests compare optimized vs unoptimized models and enabling request batching (10 ms batch timeout) yields large throughput gains at the cost of added latency.

AI-written summary · about

" chapters (8)

1. 0:00Introduction
2. 2:49High-performance TensorFlow
3. 5:57CUDA streams and GPU optimization
4. 9:33TensorFlow debugger and queues
5. 13:24Distributed TensorFlow
6. 17:22Inference with XLA
7. 24:46Model serving
8. 32:20Q&A

" related talks

• 42mHigh Performance Distributed TensorflowChris Fregly · PyData Seattle 2017
• 14mOptimize your TensorFlow Lite modelsEthan Kim, Yunlu Li · Google I/O 2021
• 59mBuilding models for TPU devicesMoisés Martínez · GDG DevFest Galicia 2019
• 21mThings I learned running neural networks on microcontrollersSaradindu Sengupta · PyData Global 2022
• 43mTensorFlow for Mobile DevelopersEnrique López Mañas · Mobile Era 2017
• 34mTensorFlow LiteNatalie Pistunovich · Mobile Era 2018
• 38mCutting Edge TensorFlow: New TechniquesElie Burzstein, Josh Dillon, Michael Bendersky, Sofien Bouaziz · Google I/O 2019
• 25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and QualityPhilip Kiely · Optimized AI Conference 2025