Expose TransitiveTestsProvider to Starlark
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the problem / feature request:
> `native.test_suite` rules only accept `tests` dependencies that are other `test_suite`s or actual `*_test` targets. It would be very helpful to have `test_suite` expose a provider for `tests` so that custom rules can be used as dependencies.
### Feature requests: what underlying problem are you trying to solve with this feature?
> Bazel test suites can be hierarchical: they can either depend on other test suites or test targets themselves. When using tags as filtration mechanisms, “filtering by specified `tags` is only done for tests listed directly in this attribute. If this attribute contains `test_suites`s, the test inside those will not be filtered by this `test_suite` (they are considered to be filtered already)”. This behavior is not ideal in all cases. Consider the following:
```
test_suite(
name=‘precheckin_tests’,
tests=[
‘//a/b/c:d’, # size=small
‘//e/f:g’, # size=medium
‘//w/x/y:z’, # size=large
],
)
# The following doesn’t work, and will run all tests in precheckin_tests, regardless of size
test_suite(
name=‘fast_precheckin_tests’,
tests=[‘:precheckin_tests’],
tags=[‘small’],
)
```
> In order to get around this issue, for each `test_suite` target, I wanted to automatically expand its `test_suite` dependencies into leaf `*_test` targets, so that filtration as above can work. To do this, I created a Bazel rule & aspect that recursively search through listed `tests` in `test_suite` and gathered all the leaf labels. This works, but *there is no way to use the new rule as a dependency of `test_suite`, because `test_suite` doesn’t expose any providers*.
```
TestInfo = provider(
fields = {
‘leaf_tests’: ‘underlying test targets for hierarchical native.test_suites`,
}
)
def _test_suite_leaf_tests_aspect_impl(target, ctx):
if hasattr(ctx.rule.attr, ‘tests’):
# Recursive test suite, gather labels of dependencies
leaf_tests = depset(
direct=[],
transitive=[t[TestInfo].leaf_tests for t in ctx.rule.attr.tests],
)
return [TestInfo(leaf_tests=leaf_tests)]
else:
# Test target, just gather its label
leaf_tests = depset(direct=[ctx.label])
return [TestInfo(leaf_tests=leaf_tests)]
test_suite_leaf_tests_aspect = aspect(
implementation=_test_suite_leaf_tests_aspect_impl,
attr_aspects=[‘tests’],
)
def _test_suite_leaf_tests_rule_impl(ctx):
leaf_tests = depset(transitive=[t[TestInfo].leaf_tests for t in ctx.attr.tests])
# Outputs correct list
print(leaf_tests.to_list())
# This cannot be used by native.test_suite :(
return [TestInfo(leaf_tests=leaf_tests)]
test_suite_leaf_tests = rule(
implementation=_test_suite_leaf_tests_rule_impl,
attrs = {
‘tests’: attr.label_list(aspects=[test_suite_leaf_tests_aspect])
}
)
```
### Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
> N/A
### What operating system are you running Bazel on?
> RHEL7
### What's the output of `bazel info release`?
> release 0.24.0- (@non-git)
### If `bazel info release` returns "development version" or "(@non-git)", tell us how you built Bazel.
> Company-specific, but behavior matches Bazel documentation
### What's the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?
> Company-specific, but behavior matches Bazel documentation
### Have you found anything relevant by searching the web?
> Documented Bazel providers (https://docs.bazel.build/versions/main/skylark/lib/skylark-provider.html) and reading source code in `bazel/src/main/java/com/google/de tools/build/lib/rules/test/*` indicates that the internal `TransitiveTestProvider` is not exposed to StarLark
### Any other information, logs, or outputs that you want to share?
> N/A
Contributor guide
Research direction
Start with native.test_suite and the test-rule sources under bazel/src/main/java/com/google/de tools/build/lib/rules/test/*, then compare the internal TransitiveTestProvider with the documented Starlark providers. Reproduce the hierarchical test_suite example and verify that a custom rule can consume the exposed provider so tag filtering reaches leaf tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100