Campo: Cost-Aware Performance Optimization for Mixed-Precision Neural Network...
summary
Campo is a tool that models per-operator FP16 performance and the cost of FP16/FP32 casts using hardware counters, then rewrites TensorFlow graphs to apply precision only when predicted benefit exceeds casting overhead. It builds regression models per operator, predicts casting cost and low-precision speedups, and dynamically rewrites graphs at runtime. Evaluation on RTX 2080 Ti and V100 shows 20.8% average throughput gain (up to 24.5%) and reduces casts by 27% without accuracy loss.
AI-written summary · about
chapters (11)
- 0:00Start
- 1:16Why mixed-precision training?
- 6:10Workload characterization
- 6:57Profiling results & observations
- 10:28Our solution: Campo
- 11:48Performance modeling
- 18:17Experimental setup
- 19:00Training throughput
- 19:24Number of cast operation nodes
- 19:52TC utilization
- 20:48Model Accuracy
related talks
05mApex
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
10mPyTorch Performance
36mHigh-Performance Distributed Tensorflow
25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and Quality
12mMixPert: Optimizing Mixed-Precision Floating-Point Emulation on GPU Integer Tensor Cores
25mFp8 Training From Hopper To Blackwell
21mThings I learned running neural networks on microcontrollers