gotestyourself / gotestyourself/gotestsum

`--rerun-fails` reruns all tests in packages if packages are also passed to `go test` args

Open Beginner friendly
#417 3 comments 0 reactions 0 assignees View on GitHub
bug docs
Dominant language
Go
Stars
2.7k
Forks
171
PR merge metrics
No merged PRs in 30d

Description

That is, if you run `gotestsum --rerun-fails --packages ./... -- ../..` then on failure all packages matching `./...` will be re-tested. I don't think this is necessarily a bug, but might warrant some more explicit documentation

Reproduction:

```console
$ go mod init module
$ mkdir --parents pkg/foo pkg/bar
```

```go
// pkg/foo/foo.go
package foo

func BoringFunc() int {
return 3
}
```

```go
// pkg/foo/foo_test.go
package foo

import (
"testing"
)

func TestBoringFunc(t *testing.T) {
if BoringFunc() != 3 {
t.Fatal("wrong")
}
}
```

```go
// pkg/bar/bar.go
package bar

func BoringFunc() int {
return 4
}
```

```go
// pkg/bar/bar_test.go
package bar

import (
"testing"
)

func TestBoringFunc(t *testing.T) {
if BoringFunc() != 3 {
t.Fatal("wrong")
}
}
```

running `gotestsum --rerun-fails --packages ./...` re-runs only the failing test, as expected:

```console
$ gotestsum --rerun-fails --packages ./... -- -count=1
✓ pkg/foo (2ms)
✖ pkg/bar (2ms)

DONE 2 tests, 1 failure in 0.188s

✖ pkg/bar (2ms)

DONE 2 runs, 3 tests, 2 failures in 0.371s

✖ pkg/bar (2ms)

=== Failed
=== FAIL: pkg/bar TestBoringFunc (0.00s)
race_test.go:9: wrong

=== FAIL: pkg/bar TestBoringFunc (re-run 1) (0.00s)
race_test.go:9: wrong

=== FAIL: pkg/bar TestBoringFunc (re-run 2) (0.00s)
race_test.go:9: wrong

DONE 3 runs, 4 tests, 3 failures in 0.539s
```

However, when run with a list of packages passed as args, _all_ tests are re-run:

```console
$ gotestsum --rerun-fails --packages ./... -- -count=1 ./...
✖ pkg/bar (2ms)
✓ pkg/foo (2ms)

DONE 2 tests, 1 failure in 0.209s

✖ pkg/bar (2ms)
✓ pkg/foo (2ms)

DONE 2 runs, 4 tests, 2 failures in 0.395s

✖ pkg/bar (2ms)
✓ pkg/foo (2ms)

=== Failed
=== FAIL: pkg/bar TestBoringFunc (0.00s)
race_test.go:9: wrong

=== FAIL: pkg/bar TestBoringFunc (re-run 1) (0.00s)
race_test.go:9: wrong

=== FAIL: pkg/bar TestBoringFunc (re-run 2) (0.00s)
race_test.go:9: wrong

DONE 3 runs, 6 tests, 3 failures in 0.556s
```

Maybe it's worth updating the docs to mention how packages should (or shouldn't) be passed when using `--rerun-fails`?

Further, I guess `gotestsum` _could_ try and determine if the user passed both `--packages` and a list of packages as args to `go test` and do something then? But I think an update to the docs would be enough.

I ran into this behaviour in a private repo at my work, but here's another example I found in the wild: https://github.com/hashicorp/nomad/blob/54aafa574d2dc4bd281b28f45dc00593c1e337da/GNUmakefile#L287

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the documentation for --rerun-fails, --packages, and arguments passed after --, then reproduce the two commands shown in the issue with the example pkg/foo and pkg/bar tests. Update the relevant guidance so users know how package arguments affect reruns, and verify that the documented behavior matches both command forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.