apache / apache/beam

Failed to construct instance from factory method SparkRunner#fromOptions

Open
#19,068 0 comments 0 reactions 0 assignees View on GitHub
bug P3 runners spark
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Unable to run Spark as a runner due to following errors- 
```

3910 [main] INFO org.spark_project.jetty.server.handler.ContextHandler - Started o.s.j.s.ServletContextHandler@18324f97{/metrics/json,null,AVAILABLE,@Spark}
Exception
in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method SparkRunner#fromOptions(interface
org.apache.beam.sdk.options.PipelineOptions)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233)

at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162)
at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)

at org.apache.beam.sdk.Pipeline.create(Pipeline.java:150)
at com.svayamtech.bean_app.Wordcount_lambda.main(Wordcount_lambda.java:42)
Caused
by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222)

... 4 more
Caused by: java.lang.NoClassDefFoundError: org/apache/beam/runners/core/construction/PipelineResources

at org.apache.beam.runners.spark.SparkRunner.fromOptions(SparkRunner.java:129)
... 9 more
Caused
by: java.lang.ClassNotFoundException: org.apache.beam.runners.core.construction.PipelineResources

at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)

at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown
Source)
... 10 more
4022 [Thread-1] INFO org.apache.spark.SparkContext - Invoking stop() from shutdown
hook
```

Imported from Jira [BEAM-4732](https://issues.apache.org/jira/browse/BEAM-4732). Original Jira may contain additional context.
Reported by: piyush69.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.