DynamoRIO / DynamoRIO/dynamorio
TEST REPORTING BUG: Wrong issue number can be reported for failing tests that are ignored
- Dominant language
- C
- Stars
- 3.2k
- Forks
- 629
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 31
Description
**Describe the bug**
Test runs can report the wrong issue number for ignored tests.
**To Reproduce**
See for example,
https://github.com/DynamoRIO/dynamorio/actions/runs/7731301531/job/21078628227#step:7:84492
```
====> FAILURE in debug-internal-64 <====
debug-internal-64: 902 tests passed, **** 6 tests failed, of which 2 were flaky, but ignoring 3 for i#2941: ****
(ignore: i#2941) code_api|client.attach_test
code_api|api.static_maps_mixup_novars_FLAKY
(ignore: i#2941) code_api,opt_memory|common.loglevel
(ignore: i#2941) code_api,opt_speed|common.decode-stress
code_api|tool.drcachesim.invariants
code_api|api.static_sideline_FLAKY
WARNING: maximum warning/error limit hit for release-external-64!
Manually verify whether it succeeded.
```
Note i#2941 is being reported as the issue for the attach_test failure.
We'll need to augment ignored test marking to let one specify the issue so that it makes it through to the test output.
Contributor guide
Assessment
This issue has not been assessed yet.