apache / apache/beam

ValueProvider with default value fails using DirectRunner

Open
#18,789 0 comments 0 reactions 0 assignees View on GitHub
bug dataflow P3 runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

 The bug is shown by the following short piece of code. 
```

public class ShowProblem implements Serializable {
  public interface MyOptions
extends
PipelineOptions, DataflowPipelineOptions {
      @Description("OutputFile")
      @Default.String("gs://xyzzy/helloWorld")
 
    ValueProvider getOutput();
      void setOutput(ValueProvider value);
    }
 
public static void main(String args[]) {
    MyOptions options =  PipelineOptionsFactory.fromArgs(args)  
            .as(MyOptions.class);
    Pipeline pipeline = Pipeline.create(options);
    pipeline
 
      .apply(Create.of("hello", "world", "!"))
        .apply(TextIO.write().to(options.getOutput()));
 
  pipeline.run();
  }
}

```

 If this pipeline is run with the "–output=gs://xyzzy/myfile" option, it runs fine both locally and on Google.  If this pipeline is run without the \--option, it runs fine on Google, but fails when run locally.

When given an output argument, the ValueProvider returned by getOutput() is a StaticValueProvider; when using the default value returns a RuntimeValueProvider,   The latter seems to have a bug hinted at in the comment just above RuntimeValueProvider.get().  The code expects that when running the pipeline (and only when running the pipeline), someone has already called
```

RuntimeValueProvider.setRuntimeOptions(options)
```

There is apparently no call to this method when using a DirectRunner. 

 

 

 

 

Imported from Jira [BEAM-4232](https://issues.apache.org/jira/browse/BEAM-4232). Original Jira may contain additional context.
Reported by: fyellin.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.