Go, ONNX and AI in Production
summary
Bramfield Trading Company moves models from Python training into ONNX files and runs live inference in a Go stack that makes up roughly 90% of production using a feature-gen service, TimeScaleDB and K3S. Classical ML (LightGBM) hits 48.6µs then 24.3µs after allocation fixes on a Dell R730 with 72GB RAM and a Tesla P40, while LLM inference takes ~1.7s and fails at trading; precomputing features fixes feature divergence and deterministic inputs make backtest, paper trade and prod comparable, no manual trading allowed.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:13From books to algorithmic trading
- 4:25Order books and trading fundamentals
- 6:31Technical indicators and strategies
- 8:17Model selection challenges
- 10:49ONNX bridge Python and Go
- 14:24Feature divergence in AI models
- 16:10Separate feature computation
- 17:52LLMs for trading experiments
- 19:17Hardware and inference benchmarks
- 21:06Optimizing Go for low-latency
- 22:50Consistent feature feeding
- 24:20Feature service and time-series DB
- 25:47GPU acceleration integration
- 26:29AI-assisted development with Cropy
- 28:58Go as AI trading backbone
related talks
36mApplied AI on the edge
01h 45mRun Frontier AI at Home
23mThe Race to the Bottom: Low Latency in the age of the Transformer
18mDeploying ML solutions with low latency in Python
36mAccelerating High-Performance Machine Learning at Scale i...
42mAI Everywhere– accelerate your development from PC to cloud
36mAccelerating Machine Learning at Scale with
29mMachine learning in production