apache / apache/beam

Figure out TestPipeline.create(PipelineOptions) / TestPipeline.fromOptions(PipelineOptions) story

Open
#18,051 0 comments 0 reactions 0 assignees View on GitHub
core improvement java P3 tests
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

TestPipeline integrates with the integration testing environment and relies heavily on being able to be configured by the environment and executed on many runners.

Tests which rely on mutating PipelineOptions before creating the TestPipeline easily can get the integration wrong by creating PipelineOptions from PipelineOptionsFactory and then calling either TestPipeline.create(options) or TestPipeline.fromOptions(options), thus ignoring any integration environment pipeline options specified.

We should fix the exposed methods on TestPipeline to prevent users from making this simple mistake.

One suggestion is to create a TestPipeline builder which will give access to a mutable PipelineOptions which the user can edit before calling build() creating a TestPipeline.

Imported from Jira [BEAM-672](https://issues.apache.org/jira/browse/BEAM-672). Original Jira may contain additional context.
Reported by: lcwik.

Contributor guide

Open the contributing guide

Research direction

Start by reading the exposed TestPipeline.create(PipelineOptions) and TestPipeline.fromOptions(PipelineOptions) entry points and how integration environment options are supplied. Determine the intended builder or API behavior, then verify that user mutations cannot bypass environment-provided options and that TestPipeline creation remains configurable across runners.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.