apache / apache/beam

failIfNoTests: causes issues when trying to run integration tests

Open
#18,437 0 comments 0 reactions 0 assignees View on GitHub
bug P3 tests
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Summary:
We have a couple different maven projects in beam that override the failIfNoTests property in ways that easily affect folks that are trying to run tests and end up having to build those projects but don't want to run tests in those projects.

I think we should remove these overrides since I think they do more harm than good. The fact that they don't allow overriding by the user is particularly tricky to get around.

Details:
projects overriding the failIfNoTests in intrusive ways are:
runners/direct/java/pom.xml (not profile protected at all)
runners/google-cloud-dataflow-java/pom.xml (when run with dataflow-runner profile)
runners/apex/pom.xml (not profile protected)

This shows up in things like perfkitbenchmarker, where if you try to run the default pkb command for beam, it is failing: (see below for repro recreating what this does)
python pkb.py \--benchmarks=beam_integration_benchmark \--beam_it_args=\--tempRoot=gs://[bucket]/staging \--beam_sdk=java

To repro:
mvn -e verify -Dit.test=org.apache.beam.examples.WordCountIT -DskipITs=false -Pdataflow-runner -DintegrationTestPipelineOptions=["--tempRoot=gs://sisk-test/staging","--runner=TestDataflowRunner"]("--tempRoot=gs://sisk-test/staging","--runner=TestDataflowRunner")

This is very reasonable command line that should work (and pkb expects it to work.)

However, this includes a specific test (-Dit.test=..), which means it will fail when it encounters the google-cloud-dataflow-java project.

cc [~davor] [~jasonkuster]

Imported from Jira [BEAM-2554](https://issues.apache.org/jira/browse/BEAM-2554). Original Jira may contain additional context.
Reported by: sisk.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the failIfNoTests settings in runners/direct/java/pom.xml, runners/google-cloud-dataflow-java/pom.xml, and runners/apex/pom.xml, including the dataflow-runner profile. Run the provided Maven verify command with the WordCountIT selector to reproduce the failure. Done means the command can select the integration test without unintended failures from those projects.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.