deeplearning4j / deeplearning4j/deeplearning4j-examples

Class EmrSparkExample - import errors

Open
#575 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
2.5k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

I imported dl4j-examples in Eclipse. I use newest code from GitHub. Class EmrSparkExample shows 51 errors. First import errors are on all import com.amazonaws.... Other errors depends on import errors.

pom.xml includes these these aws dependecies:

com.amazonaws
aws-java-sdk-emr
${aws.sdk.version}
provided


com.amazonaws
aws-java-sdk-s3
${aws.sdk.version}
provided

My IDEA doesn't show errors. But after run I get
1) first run
Error running 'EmrSparkExample'
Command line is too long. In order to reduce its length classpath file can be used.
Would you like to enable classpath file mode for all run configurations of your project?
I enabled it.
2) next run:
java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentials
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: com.amazonaws.auth.AWSCredentials
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main"
Process finished with exit code 1

In my project I try use only one dependency:

com.amazonaws
aws-java-sdk
1.11.210

I have no one error.

If I run main method I will get exception:
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.conn.ssl.SSLConnectionSocketFactory.(Ljavax/net/ssl/SSLContext;Ljavax/net/ssl/HostnameVerifier;)V
at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.(SdkTLSSocketFactory.java:56)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:92)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:315)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:299)
at com.amazonaws.AmazonWebServiceClient.(AmazonWebServiceClient.java:172)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient.(AmazonElasticMapReduceClient.java:256)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClientBuilder.build(AmazonElasticMapReduceClientBuilder.java:60)
at com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClientBuilder.build(AmazonElasticMapReduceClientBuilder.java:26)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at cz.vhr.examples.dl4j.EmrSparkExample.entryPoint(EmrSparkExample.java:103)
at cz.vhr.examples.dl4j.EmrSparkExample.main(EmrSparkExample.java:83)

Could you help?

Thank you

clavvis

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.