microsoft / microsoft/onnxruntime-genai

[Java] `onnxruntime-genai-cuda.dll` missing from Java JAR and extraction logic in CUDA builds

Open
#2,028 1 comment 0 reactions 0 assignees View on GitHub
api:java ep:CUDA
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

When building the Java bindings with CUDA support enabled, the resulting `onnxruntime-genai-[version].jar` is incomplete for GPU execution.

Specifically, the POST_BUILD commands in [`src/java/CMakeLists.txt`](https://github.com/microsoft/onnxruntime-genai/blob/639dcc06fbffff14c06612b64d3f7773a7b98a2f/src/java/CMakeLists.txt) explicitly copy `onnxruntime-genai.dll` and `onnxruntime-genai-jni.dll` to the `native-lib` staging directory, but there is no corresponding instruction to include `onnxruntime-genai-cuda.dll`.

Additionally, even if the library were present in the JAR, the [`GenAI.java`](https://github.com/microsoft/onnxruntime-genai/blob/639dcc06fbffff14c06612b64d3f7773a7b98a2f/src/java/src/main/java/ai/onnxruntime/genai/GenAI.java) source lacks the necessary logic to identify and extract the CUDA-specific binary to the temporary runtime directory, preventing the library from being loaded by the JVM.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the POST_BUILD commands in src/java/CMakeLists.txt and the native-library loading and extraction logic in src/java/src/main/java/ai/onnxruntime/genai/GenAI.java. Build the Java bindings with CUDA enabled, inspect the generated JAR and temporary runtime directory, and verify that GPU execution can load the CUDA library.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, java
Domain
backend, build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.