JVM in the Age of AI: Babylon, Valhalla, TornadoVM and friends
summary
Explains concrete JVM plumbing for ML: Project Panama jextract and the Foreign Memory API replace brittle JNI for native CUDA/CPU bindings; half-floats (float16) land in Valhalla to cut model memory; Vector API incubator and autovectorization expose SIMD for faster matrix math and already power production use cases; TornadoVM task graph compiles and schedules kernels across CPU/GPU, but memory transfer remains the dominant bottleneck; several single-file C/C++ Llama runtimes have Java bindings and some Java engines match C++ throughput.
AI-written summary · about
chapters (13)
- 0:00Introduction and AI context
- 2:57What is model inference
- 8:17GraalVM and Python interop
- 10:27Pure Java model inference
- 15:43Valhalla and half floats
- 17:52JNI and Project Panama
- 22:24Vector API and GPU basics
- 25:35CPU-based LLM implementations
- 31:15Specialized hardware for AI
- 36:13Java GPU programming approaches
- 40:28TornadoVM deep dive
- 43:40HAT and Babylon Project
- 47:28Conclusion and future outlook
related talks
47mJVM in the Age of AI: Babylon, Valhalla, TornadoVM and friends
44mJava for AI
51mYour next JVM: Panama, Valhalla, Metropolis
44mWrite Once, Accelerate Everywhere: GPU-Ready Java with TornadoVM
57mJava Next - From Amber to Valhalla, from Loom to Leyden, from Babylon to Panama
43mFrom CPU to GPU and FPGAs: Supercharging Java Applications with TornadoVM
46mTornadoVM: Supercharge your Java with GPU acceleration
43mHow to make your electrons do more in Java?