oracle-samples / oracle-samples/oracle-graph
training embedding in Entity-linking with GPU
Nobody has claimed this yet.
- Dominant language
- PLSQL
- Stars
- 56
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
I have a Nvida GPU on windows with jdk version 11.0.6 and cuda==9.0. I have successfully trained embedding on cpu with a dependency addition into build.gradle file:
compile group: 'org.nd4j',name: 'nd4j-native-platform',version:'0.9.1'
I am aware that if I want to use GPU instead of cpu as computing resource, I have to replace
compile group: 'org.nd4j',name: 'nd4j-native-platform',version:'0.9.1'
to
compile group: 'org.nd4j', name: 'nd4j-cuda-9.0-platform', version: '1.0.0-beta3'.
However, the following error occurred:
PS E:\pgx_samples_master\entity_linking> ./gradlew createEmbeddings --args="-g graph_p -i data/graph_data/graph_small.js
on"
Configuration on demand is an incubating feature.
Task :createEmbeddings
18:16:18.803 INFO : Starting embeddings creation...
18:16:18.850 INFO : Setting up embeddings creation...
18:16:18.850 INFO : Input graph: data/graph_data/graph_small.json
18:16:18.850 INFO : Creating Pgx session...
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/E:/pgx_samples_master/libs/pgx-19.3/shared-lib/third-party/guice-4.2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
18:16:20.397 INFO : Session created
18:16:20.397 INFO : Opening input graph: data/graph_data/graph_small.json...
18:16:22.365 INFO : Loaded input graph: graph_small_v, loading time: 1 sec
18:16:22.396 INFO : Creating undirected graph...
18:16:22.802 INFO : Undirected graph created, exec. time: 0 sec
Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at oracle.pgx.api.PgxFuture.get(PgxFuture.java:99)
at oracle.pgx.api.beta.mllib.DeepWalkModelBuilder.build(DeepWalkModelBuilder.java:284)
at create_embeddings.CreateEmbeddingsMain$.createModel(CreateEmbeddingsMain.scala:50)
at create_embeddings.CreateEmbeddingsMain$.main(CreateEmbeddingsMain.scala:93)
at create_embeddings.CreateEmbeddingsMain.main(CreateEmbeddingsMain.scala)
Caused by: java.lang.ExceptionInInitializerError
at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.(NativeOpExecutioner.java:53)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.lang.Class.newInstance(Class.java:584)
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:6185)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:6087)
at org.nd4j.linalg.factory.Nd4j.(Nd4j.java:201)
at oracle.pgx.mllib.backend.dl4j.Dl4jBackend.initialize(Dl4jBackend.java:25)
at oracle.pgx.mllib.dl4j.deepwalk.DeepWalkModelDl4j.(DeepWalkModelDl4j.java:59)
at oracle.pgx.mllib.dl4j.deepwalk.DeepWalkModelDl4jProvider.getModel(DeepWalkModelDl4jProvider.java:15)
at oracle.pgx.engine.mllib.DeepWalkServerModel.(DeepWalkServerModel.java:73)
at oracle.pgx.engine.instance.InstanceManager.createDeepWalkModel(InstanceManager.java:235)
at oracle.pgx.engine.CoreMlLibImpl.lambda$createDeepWalkModel$35(CoreMlLibImpl.java:301)
at oracle.pgx.engine.exec.FunctionRequest.doCall(FunctionRequest.java:28)
at oracle.pgx.engine.exec.Task.call(Task.java:326)
at oracle.pgx.engine.exec.Task.run(Task.java:246)
at oracle.pgx.engine.exec.SameThreadExecutor.execute(SameThreadExecutor.java:32)
at oracle.pgx.engine.exec.SameThreadExecutor.managedExecute(SameThreadExecutor.java:76)
at oracle.pgx.engine.exec.Task.submitTo(Task.java:230)
at oracle.pgx.engine.exec.Task.trySubmit(Task.java:210)
at oracle.pgx.engine.exec.ExecutionManager.tryDispatch(ExecutionManager.java:63)
at oracle.pgx.engine.Server.visit(Server.java:367)
at oracle.pgx.engine.Server.run(Server.java:284)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: ND4J is probably missing dependencies. For more information, please refer to: http://nd4j.org/getstarted.html
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:51)
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:19)
... 28 more
Caused by: java.lang.UnsatisfiedLinkError: no jnind4jcpu in java.library.path: [C:\Program Files\Java\jdk-11.0.6\bin, C:\WINDOWS\Sun\Java\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp, C:\Program Files\Docker\Docker\Resources\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp, C:\Windows\system32, C:\Windows, C:\Windows\System32\Wbem, C:\Windows\System32\WindowsPowerShell\v1.0, C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common, C:\Program Files (x86)\scala\bin, C:\Program Files\dotnet, H:\go\bin, C:\Program Files (x86)\Microsoft VS Code\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\WINDOWS\System32\Wbem, C:\WINDOWS\System32\WindowsPowerShell\v1.0, C:\WINDOWS\System32\OpenSSH, C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR, C:\Program Files\Java\jdk-11.0.6\bin, C:\spark-2.4.0-bin-hadoop2.7\bin, D:\MATLAB\runtime\win64, D:\MATLAB\bin, H:\Git\cmd, C:\Program Files\gradle-6.1.1\bin, C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0, D:\anaconda, D:\mysql\server\MySQL Shell 8.0\bin, C:\Users\QLR.DESKTOP-BS1O4DP\AppData\Local\Microsoft\WindowsApps, ., D:\GoLand 2019.1.1\bin, ., .]
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:976)
at org.bytedeco.javacpp.Loader.load(Loader.java:777)
at org.bytedeco.javacpp.Loader.load(Loader.java:684)
at org.nd4j.nativeblas.Nd4jCpu.(Nd4jCpu.java:10)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.bytedeco.javacpp.Loader.load(Loader.java:739)
at org.bytedeco.javacpp.Loader.load(Loader.java:684)
at org.nd4j.nativeblas.Nd4jCpu$NativeOps.(Nd4jCpu.java:63)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at org.nd4j.nativeblas.NativeOpsHolder.(NativeOpsHolder.java:29)
... 29 more
Caused by: java.lang.UnsatisfiedLinkError: no nd4jcpu in java.library.path: [C:\Program Files\Java\jdk-11.0.6\bin, C:\WINDOWS\Sun\Java\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp, C:\Program Files\Docker\Docker\Resources\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp, C:\Windows\system32, C:\Windows, C:\Windows\System32\Wbem, C:\Windows\System32\WindowsPowerShell\v1.0, C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common, C:\Program Files (x86)\scala\bin, C:\Program Files\dotnet, H:\go\bin, C:\Program Files (x86)\Microsoft VS Code\bin, C:\WINDOWS\system32, C:\WINDOWS, C:\WINDOWS\System32\Wbem, C:\WINDOWS\System32\WindowsPowerShell\v1.0, C:\WINDOWS\System32\OpenSSH, C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR, C:\Program Files\Java\jdk-11.0.6\bin, C:\spark-2.4.0-bin-hadoop2.7\bin, D:\MATLAB\runtime\win64, D:\MATLAB\bin, H:\Git\cmd, C:\Program Files\gradle-6.1.1\bin, C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0, D:\anaconda, D:\mysql\server\MySQL Shell 8.0\bin, C:\Users\QLR.DESKTOP-BS1O4DP\AppData\Local\Microsoft\WindowsApps, ., D:\GoLand 2019.1.1\bin, ., .]
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:976)
at org.bytedeco.javacpp.Loader.load(Loader.java:765)
I have cuda 9.0 toolkit in my windows PATH, could you help to look at this error?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ND4J dependency entries in build.gradle and the create_embeddings.CreateEmbeddingsMain.scala entry point used by the createEmbeddings task. Review the reported DeepWalkModelBuilder and NativeOpsHolder stack trace, then verify whether the configured backend can load successfully on the stated Windows, JDK, and CUDA setup. Done means the embedding task starts without the reported UnsatisfiedLinkError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100