apache / apache/beam

PubsubIO.fromSubscription(ValueProvider) throws RuntimeException

Open
#18,556 0 comments 0 reactions 0 assignees View on GitHub
bug gcp io java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

The very simple Pipeline:

public class PubsubTemplate {
public interface PubsubTemplateOptions extends PipelineOptions

{ ValueProvider getSubscription(); void setSubscription(ValueProvider value); }

public static void main(String[] args) throws IOException

{ PubsubTemplateOptions options = PipelineOptionsFactory.fromArgs(args).withValidation() .as(PubsubTemplateOptions.class); Pipeline p = Pipeline.create(options); p.apply(PubsubIO.readStrings(). fromSubscription(options.getSubscription())); p.run().waitUntilFinish(); }

}

Throws the exception when generating a Dataflow template:

java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Not called from a runtime context.
at org.apache.beam.sdk.options.ValueProvider$RuntimeValueProvider.get(ValueProvider.java:223)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get(ValueProvider.java:131)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get(ValueProvider.java:131)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.getSubscription(PubsubUnboundedSource.java:1374)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource$PubsubSource.(PubsubUnboundedSource.java:1103)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.expand(PubsubUnboundedSource.java:1407)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubUnboundedSource.expand(PubsubUnboundedSource.java:110)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:514)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:454)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.expand(PubsubIO.java:730)
at org.apache.beam.sdk.io.gcp.pubsub.PubsubIO$Read.expand(PubsubIO.java:536)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:514)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:454)
at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:165)
at com.google.pubsubtemplate.PubsubTemplate.main(PubsubTemplate.java:34)
... 6 more

Imported from Jira [BEAM-2965](https://issues.apache.org/jira/browse/BEAM-2965). Original Jira may contain additional context.
Reported by: bmenasha.

Contributor guide

Open the contributing guide

Research direction

Start with the PubsubIO.readStrings().fromSubscription(...) entry point and inspect PubsubUnboundedSource.getSubscription at line 1374, using the provided Dataflow template example to reproduce the failure. Done means template generation accepts the ValueProvider subscription without throwing "Not called from a runtime context."

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.