gotestyourself / gotestyourself/gotestsum
`--rerun-fails` reruns all tests in packages if packages are also passed to `go test` args
- Dominant language
- Go
- Stars
- 2.7k
- Forks
- 171
- PR merge metrics
- No merged PRs in 30d
Description
That is, if you run `gotestsum --rerun-fails --packages ./... -- ../..` then on failure all packages matching `./...` will be re-tested. I don't think this is necessarily a bug, but might warrant some more explicit documentation
Reproduction:
```console
$ go mod init module
$ mkdir --parents pkg/foo pkg/bar
```
```go
// pkg/foo/foo.go
package foo
func BoringFunc() int {
return 3
}
```
```go
// pkg/foo/foo_test.go
package foo
import (
"testing"
)
func TestBoringFunc(t *testing.T) {
if BoringFunc() != 3 {
t.Fatal("wrong")
}
}
```
```go
// pkg/bar/bar.go
package bar
func BoringFunc() int {
return 4
}
```
```go
// pkg/bar/bar_test.go
package bar
import (
"testing"
)
func TestBoringFunc(t *testing.T) {
if BoringFunc() != 3 {
t.Fatal("wrong")
}
}
```
running `gotestsum --rerun-fails --packages ./...` re-runs only the failing test, as expected:
```console
$ gotestsum --rerun-fails --packages ./... -- -count=1
✓ pkg/foo (2ms)
✖ pkg/bar (2ms)
DONE 2 tests, 1 failure in 0.188s
✖ pkg/bar (2ms)
DONE 2 runs, 3 tests, 2 failures in 0.371s
✖ pkg/bar (2ms)
=== Failed
=== FAIL: pkg/bar TestBoringFunc (0.00s)
race_test.go:9: wrong
=== FAIL: pkg/bar TestBoringFunc (re-run 1) (0.00s)
race_test.go:9: wrong
=== FAIL: pkg/bar TestBoringFunc (re-run 2) (0.00s)
race_test.go:9: wrong
DONE 3 runs, 4 tests, 3 failures in 0.539s
```
However, when run with a list of packages passed as args, _all_ tests are re-run:
```console
$ gotestsum --rerun-fails --packages ./... -- -count=1 ./...
✖ pkg/bar (2ms)
✓ pkg/foo (2ms)
DONE 2 tests, 1 failure in 0.209s
✖ pkg/bar (2ms)
✓ pkg/foo (2ms)
DONE 2 runs, 4 tests, 2 failures in 0.395s
✖ pkg/bar (2ms)
✓ pkg/foo (2ms)
=== Failed
=== FAIL: pkg/bar TestBoringFunc (0.00s)
race_test.go:9: wrong
=== FAIL: pkg/bar TestBoringFunc (re-run 1) (0.00s)
race_test.go:9: wrong
=== FAIL: pkg/bar TestBoringFunc (re-run 2) (0.00s)
race_test.go:9: wrong
DONE 3 runs, 6 tests, 3 failures in 0.556s
```
Maybe it's worth updating the docs to mention how packages should (or shouldn't) be passed when using `--rerun-fails`?
Further, I guess `gotestsum` _could_ try and determine if the user passed both `--packages` and a list of packages as args to `go test` and do something then? But I think an update to the docs would be enough.
I ran into this behaviour in a private repo at my work, but here's another example I found in the wild: https://github.com/hashicorp/nomad/blob/54aafa574d2dc4bd281b28f45dc00593c1e337da/GNUmakefile#L287
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the documentation for --rerun-fails, --packages, and arguments passed after --, then reproduce the two commands shown in the issue with the example pkg/foo and pkg/bar tests. Update the relevant guidance so users know how package arguments affect reruns, and verify that the documented behavior matches both command forms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100