bazel-contrib / bazel-contrib/rules_bazel_integration_test

Support passing arguments to the test runner

Open
#252 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Starlark
Stars
65
Forks
16
Avg merge
1d 9h
Merged PRs (30d)
9

Description

I want to create my own macro around using the `rules_bazel_integration_test` and I created a custom test runner in python that uses `pytest`. I want to be able to specify a default set of "tests" required to pass on each target type, eg.: "bazel info" and "bazel query //..."

I would also like to have each caller of the custom macro specify unique tests. My macro is essentially this

```python

def __get_label(label):
"""
Return a Label object representation.

Args:
label: a string label"""
if label.startswith("@") or label.startswith("//"):
return label
else:
return Label("//{}:{}".format(native.package_name(), label.lstrip(":")))

def __bazel_py_intetration_test(
name,
srcs,
workspace_path,
run_files,
main = None,
tags = None,
deps = [],
env = {},
target_compatible_with = None):
test_scenario_name = "{}_scenario".format(name)
tags = tags or []
srcs = [__get_label(src) for src in srcs] + [
"@//bazel/integration_tests:bit_python_runner.py", ## The test runner py_library
"@//bazel/integration_tests/py:common_py_bit_tests", # common tests
]
python.binary(
name = test_scenario_name,
main = ":bit_python_runner.py",
srcs = srcs,
args = [
"--fixtures-per-test",
"-rpa",
"-vvvv",
] + ["$(locations {})".format(src) for src in srcs],
deps = deps+[
# any dependencies required for :bit_python_runner.py
],
testonly = True,
)

workspace_files = integration_test_utils.glob_workspace_files(workspace_path)

workspace_files = [
"@//:local_repository_files", # all files in the main workspace
] + workspace_files + run_files

# Declare the integration tests.
bazel_integration_test(
name = name,
bazel_binary = "@//tools:bazel",
test_runner = ":{}".format(test_scenario_name),
env = env,
tags = ["exclusive", "bit"] + tags,
workspace_files = workspace_files,
workspace_path = workspace_path,
target_compatible_with = target_compatible_with,
)
```

The `args` value of the `python.binary` does not get called when invoking the `bazel_integration_test`.

My end goal is to be able to define a test once, and have said test execute against all "bazel_integration_test" targets. One option is to allow passing arguments to the test runner

Contributor guide

Open the contributing guide

Research direction

Start with the custom macro shown in the issue and trace how bazel_integration_test invokes its test_runner, especially whether runner arguments are accepted or forwarded. Done means callers can provide unique arguments while the macro supplies defaults, and those arguments reach the Python pytest runner for every integration-test target.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.