[6.2.0] instrumentation_filter doesn't work for bazel coverage with py_test
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the bug:
we run `bazel coverage` over our py_test target `common_tools_test`. Some of our test cases will create temp directory under execution and delete afterwards. Unexpected, this temp directory will be included into the `pylcov.dat` report which causes `no source code` error. Since on macos, pytest tmpdir will create the directory under `/private/var/folders`, I tried to omit this directory with `instrumentation_filter` but it doesn't work.
`Converting coveragepy database to lcov: /private/var/tmp/_bazel_nlou/b43af9baf9d056244183c6c3d6dd8487/sandbox/darwin-sandbox/5/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/_coverage/common_tools_test/test/pylcov.dat `
```
TN:
SF:/private/var/folders/6m/nl3wnqp96hz5b53qpx2y71f40000gp/T/pytest-of-nlou/pytest-375/test_load_config_from_fs0/dev/github.json
```
```
py_test(
name = "common_tools_test",
srcs = glob(["test/**/*.py"]),
data = glob(["test/common_tools/data/*"]) + ["//:config"],
main = "__test__.py",
python_linters = [
],
deps = [
"//:common_tools_lib",
"//confluent/config",
requirement("click"),
requirement("mock"),
requirement("pytest"),
],
)
```
```
% cat .bazelrc
# coverage --combined_report=lcov
coverage --test_env=VERBOSE_COVERAGE=true
coverage --java_runtime_version=remotejdk_11
coverage --action_env=VERBOSE_COVERAGE=true
coverage --sandbox_debug
coverage --test_output=streamed
coverage --verbose_failures
coverage --instrument_test_targets=false
coverage --instrumentation_filter="/confluent[/:],-/private/var/folders"
coverage --combined_report=none
```
Also when I run `coveragepy` tool , I am able to make this work with this configuration in `.coveragerc` .
```
[run]
omit =
.git/*
.tox/*
docs/*
setup.py
test/*
tests/*
bazel-*/*
/private/var/folders/*. <====
```
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
create a test case with pytest which uses `tmpdir` to create a temp directory and files under that directory.
e.g.
```
def test_load_config_from_fs(tmpdir, getpass):
getpass.getuser.return_value = "mzheng"
paths_json = tmpdir.mkdir("dev").join("paths.json")
paths_json.write('{"TMP_PATH": "/tmp/{{ user }}"}')
github_json = tmpdir.join("dev", "github.json")
github_json.write('{"mzheng": "maxzheng"}')
with tmpdir.as_cwd():
paths = load_config(config_name="paths", fs_root=".")
assert paths == {"TMP_PATH": "/tmp/maxzheng"}
```
### Which operating system are you running Bazel on?
darwin-x86_64
### What is the output of `bazel info release`?
bazel info release INFO: Invocation ID: da725c55-1dd1-4435-bc7c-cbe83b030e23 release 6.2.0
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
```text
git@github.com:confluentinc/common-tools.git
```
### Have you found anything relevant by searching the web?
no
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Start with the .bazelrc coverage settings and the py_test target common_tools_test; reproduce with bazel coverage and a pytest tmpdir test, then inspect the generated pylcov.dat. Done means the instrumentation_filter behavior excludes the temporary directory without producing a "no source code" error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100