Failed to construct instance from factory method SparkRunner#fromOptions
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Unable to run Spark as a runner due to following errors-
```
3910 [main] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@18324f97{/metrics/json,null,AVAILABLE,@Spark}
Exception
in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method SparkRunner#fromOptions(interface
org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233)
at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
at org.apache.beam.sdk.Pipeline.create(Pipeline.java:150)
at com.svayamtech.bean_app.Wordcount_lambda.main(Wordcount_lambda.java:42)
Caused
by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222)
... 4 more
Caused by: java.lang.NoClassDefFoundError: org/apache/beam/runners/core/construction/PipelineResources
at org.apache.beam.runners.spark.SparkRunner.fromOptions(SparkRunner.java:129)
... 9 more
Caused
by: java.lang.ClassNotFoundException: org.apache.beam.runners.core.construction.PipelineResources
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown
Source)
... 10 more
4022 [Thread-1] INFO org.apache.spark.SparkContext - Invoking stop() from shutdown
hook
```
Imported from Jira [BEAM-4732](https://issues.apache.org/jira/browse/BEAM-4732). Original Jira may contain additional context.
Reported by: piyush69.
Contributor guide
Assessment
This issue has not been assessed yet.