Tokio, Rayon, and the GPU: Profiling a Radiology AI Pipeline in Rust
Talkat RustConf 2026 · on Wednesday, 2026-09-09· at 14:00· Room 511 | Palais des Congrès de Montréal, Palais des Congrès de Montréal·
summary
A radiology pipeline ports data loading and live inference to Rust, runs a homegrown 1 billion-parameter LLM, and processes thousands of images per day on a commodity GPU; training consumes ~200 TB of CT data and scans it in ~24 hours (~2.3 GB/s). Latency targets are <5s for X‑rays and <30s for CTs; trials show up to 80% efficiency gains. The stack uses Tokio and Rayon, instruments with tracing + perf, and fixes scheduling by bounded reordering and disabling Rayon’s global parallelism.
AI-written summary · about
chapters (10)
- 0:00Introduction to Aries pipeline
- 3:57Team and project origin
- 7:48AI pipeline architecture overview
- 10:36Python implementation challenges
- 13:43Rust port and benefits
- 17:33Profiling for performance gains
- 21:43Tracing crate for async streams
- 27:18Rayon scheduler inefficiencies
- 34:02Final optimizations and results
- 34:22Q&A session
related talks
01h 00mFrom Tiny Chips to Server Racks: Deploying AI Everywhere with Rust
23mThe Race to the Bottom: Low Latency in the age of the Transformer
16mEmpowering Fearless Data Engineering
25m"Rust for AI & Accelerated Computing"
46mPath of Data & AI Challenges
31mHow Rad AI is Revolutionizing Radiology with Ray
55mLarge Scale Distributed AI on the JVM
29mLessons Learned: Porting a Streaming Data Pipeline from Scala to Rust.