tensorflow / tensorflow/java

Compiling from source, cuDNN version is not compatible? How can I change the cuDNN compile version?

Open
#500 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
928
Forks
227
PR merge metrics
No merged PRs in 30d

Description

I use mvn install java-0.5.0 in jetson with below environment. Finally it completed and generated the "tensorflow-core-api-0.5.0-linux-arm64-gpu.jar".

System information

  • OS Platform and Distribution : Linux Ubuntu 20.04 arm64
  • TensorFlow installed from (source or binary): source
  • TensorFlow version: 2.10.1
  • Java version (i.e., the output of java -version): openjdk version "11.0.20"
  • Java command line flags (e.g., GC parameters):
  • Installed from Maven Central?:
  • Bazel version (if compiling from source): 5.4.1
  • GCC/Compiler version (if compiling from source): 9.4.0
  • CUDA/cuDNN version: CUDA11.4+CuDNN8.6.0
  • GPU model and memory:

But when I run the model , the error is occured. Here is the error log:
"2023-08-01 09:57:03.793190: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_dnn.cc:377] Loaded runtime CuDNN library: 8.6.0 but source was compiled with: 8.9.0. CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration."
453ad32b83ae0637ea2e646e0504851

I have tried upgrade CuDNN to 8.9.0 and run again, but it's run failed with error-"tensorflow/core/framework/op_kernel.cc:1780] OP_REQUIRES failed at conv_ops.cc:1143 : NOT_FOUND: No algorithm worked! Error messages:". I think this may be the environment doesn't match.
614c484dff4a18d1c0a9bd186efcfa2

So how can I resolve this problem? I think this is a way to fix the problem that is to change the cuDNN compile version, but I can't find any info about this.
P.S.: I run the model successfully in my local machine(windows11 x86_64...)

Best Regards

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the mvn install java-0.5.0 build and model run on the reported Jetson environment, comparing the cuDNN version used during compilation with the runtime version. Done means the generated GPU JAR runs the model successfully with compatible CUDA and cuDNN versions, or the required compile configuration is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.