bazelbuild / bazelbuild/bazel

test_suite doesn't honor target_compatible_with

Open
#29,137 1 comment 0 reactions 0 assignees View on GitHub
P3 team-Rules-API type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

I have a patchwork of platform-specific test suites (in rules_foreign_cc); i would like to somehow make it so bazel test //... hits the right test_suite, like:

```python

test_suite(
name = "linux_suite",
target_compatible_with = ["@platforms//os:linux"],
tests = ["@otherrepo//:test1", "@otherrepo//:test2"],
)

test_suite(
name = "windows_suite",
target_compatible_with = ["@platforms//os:windows"],
tests = ["@otherrepo//:test1"],
)
```

The actual setup here is a little more annoying, which is why I don't want to have to put target_compatible_with on all of the individual other targets; instead, they just live in a different repo. However, I discovered that this doesn't actually work because target_compatible_with doesn't do anything here. @katre and @pcjanzen requested I file this, though it took me a few weeks!

### Which category does this issue belong to?

_No response_

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

https://github.com/novas0x2a/repro-test-suite-compatible

Just run `bazel test //...`

### Which operating system are you running Bazel on?

linux

### What is the output of `bazel info release`?

release 9.0.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

I don't think it's a regression.

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

We had a small slack thread about it: https://bazelbuild.slack.com/archives/CA31HN1T3/p1773794349966379?thread_ts=1773736456.062499&cid=CA31HN1T3

There are a variety of ways that (in theory) could be used to implement this same concept; unfortunately, none of them work on test_suites today:
- target_compatible_with (see above)
- an alias that selects the right suite (I think all paths are followed here?)
- a select on test_suite.tests (since it needs to be available without configuration)

Contributor guide

Open the contributing guide

Research direction

Start with the linked repro repository and run `bazel test //...` on Linux with Bazel 9.0.1. Trace how `test_suite` handles `target_compatible_with`, its `tests` labels, aliases, and `select`; done when the platform-compatible suite is selected without adding compatibility constraints to the individual tests.

Written by the indexing model from the issue text.

Assessment

Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.