Lifting PT to New Heights with MAX and Mojo
summary
Modular builds the MAX inference engine plus a new language, Mojo, to ship a Mojo kernels library with three abstraction layers (assembly-like, CUDA-like, high-level typed kernels) aimed to beat Triton’s ~90% to 100%+ performance. Runtime executes a generic graph IR and accepts TorchScript, ONNX and FX frontends. Initial shortcut of converting FX to TorchScript works but drops intermediate shape info, forcing a move to native FX graphs and a custom torch.compile backend.
AI-written summary · about
related talks
28mMojo & MAX - A modern AI-first SW Development ecosystem
25mMojo + PyTorch: A Simpler, Faster Path To Custom Kernels
11mIs Mojo programming language just a hype?
24mAdvancements in High-Performance AI/ML through PyTorch's Python Compiler
12mLightning Talk: Accelerating Inference on CPU with Torch.Compile
11mAccelerated Software for a Post-Moore World
24mInside MoE Optimization: A Profiler-Guided Tour of torch.compile and vLLM
38mFireside chat