`--test_output=streamed` disables validation for subsequent `--test_strategy` flag
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 72
Description
### Description of the problem / feature request / question:
If I use `--test_output=streamed`, then pass `--test_strategy=X`, Bazel doesn't give a warning about the invalid strategy, nor does it indicate that it'll ignore the flag.
If I change `streamed` to `errors`, or remove the `--test_output` flag, then Bazel correctly errors out that `X` is not a valid strategy.
I guess the reason is [`--test_output=streamed` implies `--test_strategy=exclusive`](), but unless you know this, it can be confusing to expect for example a sandboxed test execution but get an exclusive (non-sandboxed) one, without any warning from Bazel.
### If possible, provide a minimal example to reproduce the problem:
```
$ cat BUILD
sh_test(
name = 'foo_test',
srcs = ['foo_test.sh'],
)
$ cat foo_test.sh
#!/bin/bash
$ bazel test //:foo_test --test_output=streamed --test_strategy=X
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
INFO: Analysed target //:foo_test (0 packages loaded).
INFO: Found 1 test target...
Target //:foo_test up-to-date:
bazel-bin/foo_test
INFO: Elapsed time: 0.168s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action
//:foo_test (cached) PASSED in 0.1s
Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
$ bazel test //:foo_test --test_output=errors --test_strategy=X
ERROR: 'X' is an invalid value for test strategy. Valid values are: exclusive, experimental_worker, standalone
INFO: Elapsed time: 0.140s
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Couldn't start the build. Unable to run tests
$ bazel test //:foo_test --test_strategy=X
ERROR: 'X' is an invalid value for test strategy. Valid values are: exclusive, experimental_worker, standalone
INFO: Elapsed time: 0.135s
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Couldn't start the build. Unable to run tests
```
### Environment info
* Operating System:
Ubuntu 14.04
* Bazel version (output of `bazel info release`):
0.8.0
Contributor guide
Research direction
Start by reproducing the issue with the two Bazel test commands using --test_output=streamed and --test_strategy=X, then inspect Bazel's command-line option parsing and test-strategy validation. Done means the invalid strategy is rejected or clearly warned about even when streamed output implies exclusive execution, with regression coverage for the shown combination.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, cli, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100