Issue while executing scala spark code
- Ngôn ngữ chính
- Java
- Star
- 6.8k
- Fork
- 1.6k
- Merge trung bình
- 44 phút
- Pull request đã merge (30 ngày)
- 1
Mô tả
Hello team,
I am a new to spark and done some trainings recently along with my colleagues. There are set of precompiled codes which are executing for others however i am getting below error message:
actions taken: deleted repository folder under .m2 folder. and restarted eclipse. all errors resolved after build path is finished.
however while running any code i am getting almost similar error.
ERROR MESSAGE:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
21/10/01 13:53:30 INFO SparkContext: Running Spark version 2.1.1
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:38)
at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.(DefaultMetricsSystem.java:36)
at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:99)
at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:192)
at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2391)
at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2391)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2391)
at org.apache.spark.SparkContext.(SparkContext.scala:295)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
at tekcrux.DataframeJoins$.delayedEndpoint$tekcrux$DataframeJoins$1(DataframeJoins.scala:10)
at tekcrux.DataframeJoins$delayedInit$body.apply(DataframeJoins.scala:5)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at tekcrux.DataframeJoins$.main(DataframeJoins.scala:5)
at tekcrux.DataframeJoins.main(DataframeJoins.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 25 more
Thank you in advance!
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.