TornadoVM 7.3 bridges Java and Nvidia GPUs for machine learning workloads

New version introduces CUDA library integration, multi-GPU support, and a revamped compiler interface

By LineZotpaper

Published21 October 2026

Read Time2 min

The TornadoVM project has reached version 7.3, introducing significant architectural changes designed to let Java developers tap into Nvidia GPU acceleration for machine learning and data-parallel workloads. The update swaps the Java Native Interface (JNI) for the Foreign Function & Memory (FFM) API and adds direct integration with Nvidia's CUDA libraries, alongside a new inference engine called jitLLM.

TornadoVM is an open-source framework that works alongside existing JVM distributions to automatically offload Java code to hardware accelerators like GPUs. Version 7 aims to make the JVM a more viable platform for modern AI and cloud computing tasks.

The release replaces the previous JNI dependency with the FFM API and has removed the experimental JVMCI dependency. It now sources compiler metadata through reflection and an Unsafe layer, providing a uniform path across JDK versions 21, 25, 26, and 27. A headline feature is a deeply optimized CUDA backend that allows developers to call libraries such as cuBLAS and CUTLASS directly from TornadoVM TaskGraphs. This enables the swapping of JIT-generated kernels for hand-tuned Nvidia library calls to improve performance.

Version 7.3 introduces a new cuTile programming model via the TileContext for Nvidia hardware, along with multi-GPU execution support and similar integration for Apple's MXL library. The project aims to provide an idiomatic Java environment for accessing Nvidia's tuned libraries for linear algebra, Fast Fourier Transforms, and deep neural networks.

The framework integrates with tools including Apache Flink, Quarkus, and Spring AI. According to the team, a developer using Flink's Table API could use a configuration flag to signal a query for GPU acceleration, with the runtime evaluating the query's intermediate representation for suitability. TornadoVM offers several approaches for GPU programming, including a parallel annotation API, a kernel context API, and the new TileContext.

Alongside the TornadoVM release, the team introduced jitLLM v2.0, a Java-based AI inference engine built on TornadoVM. It is positioned as a direct competitor to llama.cpp and Ollama. The technology is being commercialized through a spin-off company, Beestream.ai.

Analysis

Why This Matters

Allows the massive existing Java developer ecosystem to accelerate compute-intensive tasks including AI without switching languages or stacks.
Provides a pathway for integrating GPU-accelerated AI inference directly into Java-based enterprise applications.
The ability to swap JIT kernels for optimized CUDA library calls could offer significant performance gains for production workloads.

Background

Java has historically lacked first-class support for general-purpose GPU computing compared to languages like Python, C++, or Julia. TornadoVM is an open-source project aiming to fill this gap by acting as a runtime-level bridge between the JVM and hardware accelerators. It initially used OpenCL and Apple Metal, and version 7 deepens its focus on Nvidia's CUDA ecosystem and Nvidia's optimized libraries.

Key Perspectives

TornadoVM Team: The team positions TornadoVM 7.3 and jitLLM as a major step toward making Java a first-class language for AI. By offering a write-once-run-anywhere approach for ML workloads and providing direct access to Nvidia's highly tuned libraries, they aim to give Java developers a high-performance option without leaving the JVM. Enterprise Java Developers: This community could benefit from the ability to accelerate data processing and machine learning inference within their existing tech stack, particularly in areas like real-time analytics and signal processing. Critics/Skeptics: Potential concerns include the performance overhead of the TornadoVM abstraction layer compared to writing native CUDA or using established Python AI frameworks. The maturity of the tooling and the breadth of supported operations remain key factors for wider adoption.

What to Watch

Performance benchmarks comparing jitLLM against llama.cpp and Ollama on standard inference tasks.
Adoption of TornadoVM within major Java frameworks like Flink and Spring AI.
The growth and success of the commercial entity Beestream.ai in supporting the project.

Sources

TornadoVM 7 Enables GPU-Native Workloads From Java Applications · InfoQ

Checked

12 claims in this story checked against 1 source: 7 supported.

Show each claim

SupportedTornadoVM 7.3 connects Java workloads with Nvidia GPU acceleration for machine-learning tasks.InfoQ: “The release also deepens integration with the Nvidia ecosystem through an optimized CUDA backend.”
SupportedVersion 7.3 introduces CUDA library integration.InfoQ: “This includes support for the cuTile programming model (via a new TileContext) and the ability to call CUDA libraries such as cuBLAS and CUTLASS directly from within TornadoVM TaskGraphs .”
SupportedVersion 7.3 adds multi-GPU support.InfoQ: “Finally, support for multi-GPU execution has been added.”
Not in the sourcesThe release includes a revamped compiler interface.InfoQ
SupportedThe architectural changes are designed to let Java developers use Nvidia GPU acceleration for machine-learning and data-parallel workloads.InfoQ: “This makes it a useful building block for modern cloud and machine learning workloads.”
SupportedThe backends replace JNI with FFM, and the project removes its experimental JVMCI dependency.InfoQ: “The backends now use the Foreign Function & Memory (FFM) API in place of the Java Native Interface (JNI). The experimental JVM Compiler Interface (JVMCI) dependency has also been removed.”
Partly supportedTornadoVM sources compiler metadata through reflection and an Unsafe layer, using a uniform path across JDK versions 21, 25, 26 and 27.InfoQ: “According to the team, TornadoVM now sources "all compiler metadata through a reflection + ASM + Unsafe layer" which is the single, uniform path on every supported JDK (21, 25, 26 and 27).”
Partly supportedVersion 7.3 adds a new inference engine called jitLLM.InfoQ: “Alongside the 7.3 release, the TornadoVM team introduced jitLLM v2.0, its next-generation Java AI inference engine built on TornadoVM.”
Partly supportedDevelopers can replace JIT-generated kernels with hand-tuned Nvidia library calls to improve performance.InfoQ: “Developers can swap JIT-generated kernels for CUDA library calls, or chain the two together, depending on their performance and portability needs.”
SupportedVersion 7.3 adds cuTile through TileContext and similar integration for Apple’s MXL library.InfoQ: “This includes support for the cuTile programming model (via a new TileContext)”
SupportedFor a Flink Table API query, a configuration flag can indicate potential GPU acceleration, and the query’s intermediate representation is assessed for suitability.InfoQ: “A configuration flag is used to indicate that this query should potentially be accelerated on a GPU.”
Partly supportedThe team positions jitLLM as a direct competitor to llama.cpp and Ollama.InfoQ: “The team positions jitLLM as a direct competitor to llama.cpp and Ollama in performance, and it ships as a single JAR file.”