test_suite doesn't honor target_compatible_with
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the bug:
I have a patchwork of platform-specific test suites (in rules_foreign_cc); i would like to somehow make it so bazel test //... hits the right test_suite, like:
```python
test_suite(
name = "linux_suite",
target_compatible_with = ["@platforms//os:linux"],
tests = ["@otherrepo//:test1", "@otherrepo//:test2"],
)
test_suite(
name = "windows_suite",
target_compatible_with = ["@platforms//os:windows"],
tests = ["@otherrepo//:test1"],
)
```
The actual setup here is a little more annoying, which is why I don't want to have to put target_compatible_with on all of the individual other targets; instead, they just live in a different repo. However, I discovered that this doesn't actually work because target_compatible_with doesn't do anything here. @katre and @pcjanzen requested I file this, though it took me a few weeks!
### Which category does this issue belong to?
_No response_
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
https://github.com/novas0x2a/repro-test-suite-compatible
Just run `bazel test //...`
### Which operating system are you running Bazel on?
linux
### What is the output of `bazel info release`?
release 9.0.1
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
```
### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.
I don't think it's a regression.
### Have you found anything relevant by searching the web?
_No response_
### Any other information, logs, or outputs that you want to share?
We had a small slack thread about it: https://bazelbuild.slack.com/archives/CA31HN1T3/p1773794349966379?thread_ts=1773736456.062499&cid=CA31HN1T3
There are a variety of ways that (in theory) could be used to implement this same concept; unfortunately, none of them work on test_suites today:
- target_compatible_with (see above)
- an alias that selects the right suite (I think all paths are followed here?)
- a select on test_suite.tests (since it needs to be available without configuration)
Contributor guide
Research direction
Start with the linked repro repository and run `bazel test //...` on Linux with Bazel 9.0.1. Trace how `test_suite` handles `target_compatible_with`, its `tests` labels, aliases, and `select`; done when the platform-compatible suite is selected without adding compatibility constraints to the individual tests.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100