gotestyourself / gotestyourself/gotestsum

gotestsum --rerun-fails can exit successfully after incomplete parent test run

Open
#565 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
2.7k
Forks
171
PR merge metrics
No merged PRs in 30d

Description

## Summary

`--rerun-fails` can report success after Go aborts part of a subtest tree with a parent-test `runtime.Goexit`.

This happens when a subtest calls `FailNow`/`Fatal` on a parent `*testing.T`. Go emits:

`test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test`

At that point, sibling subtests may never run. gotestsum then reruns only the failed tests it observed. If those reruns pass, gotestsum exits 0 even though the original selected test tree did not fully execute.

## Why This Is Unsafe

This is similar to the existing suspected-panic guard: the first run is incomplete, so the set of failed tests is not a safe rerun set.

The final output can look like:

```text
DONE 2 runs, 9 tests, 3 failures in 213.360s
```

but the process still exits 0 because the rerun passed.

## Expected Behavior

When gotestsum sees the parent-test `runtime.Goexit` diagnostic, `--rerun-fails` should abort with a non-zero exit rather than rerunning only observed failures.

## Related

- https://github.com/gotestyourself/gotestsum/issues/290
- https://github.com/golang/go/issues/45508

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing suspected-panic guard and the handling of Go's parent-test runtime.Goexit diagnostic. Add a regression test covering --rerun-fails with an incomplete subtest tree, then verify that the command exits non-zero instead of rerunning only the observed failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.