A torch.fx Based Compression Toolkit Empowered by torch_musa
summary
NeuroTrim is a lightweight FX-based compression toolkit that runs on native PyTorch via torch_musa and the MUSA backend, producing quantized models that can export to ONNX or custom accelerators. It combines FX, torchInductor and ExportedIR with a graph-fallback tracing strategy (symbolic, FX, export) to obtain a full graph, adds lowered operators and kernel fusions (conv+bn+activation), supports int8/int4 and float8, performs on-the-fly weight updates including GPTQ-style corrections, and reports a real case quantization of a 671B model that took days and required fault-tolerance checkpoints.
AI-written summary · about
related talks
11mWhat's New in Torch.onnx
05mPyTorch ONNX Export Support
06mTorchVision
14mLightning Talk: Every Millisecond Counts: The Fine-tuning Journey of an Ultra-Eff...
10mExecuTorch on Microcontrollers: Deploying PyTorch To...
14mOptimize your TensorFlow Lite models
10mQuantization
13mRunning ExecuTorch Applications With Silicon Accelera...