gotestyourself / gotestyourself/gotestsum
Test cases passed but gotestsum print out Failed in summary
- Dominant language
- Go
- Stars
- 2.7k
- Forks
- 171
- PR merge metrics
- No merged PRs in 30d
Description
We tried to adopt gotestsum to https://github.com/karmada-io/karmada repo, and in both CI and local tests, the summary printed by gotestsum returned FAILED even when all test cases passed. Rerun test cases again using `go test` command and it passed normally.
This is the stdout printed by gotestsum in CI (Github Actions)
ref: https://github.com/karmada-io/karmada/actions/runs/8752542558/job/24020602781?pr=4850#step:4:8673
This is the stdout printed by gotestsum in the local
```console
$ gotestsum -- -v -run=TestGetMinTolerationTime pkg/util/helper/*.go
✓ command-line-arguments (478ms)
=== Failed
=== FAIL: command-line-arguments TestGetMinTolerationTime (unknown)
-1ns=== RUN TestGetMinTolerationTime/no_usedTolerations
0s=== RUN TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations
59.999932917s=== RUN TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil
-1ns=== RUN TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil
-1ns=== RUN TestGetMinTolerationTime/find_the_latest_trigger_time
59.999909167s=== RUN TestGetMinTolerationTime/trigger_time_is_up
0s--- PASS: TestGetMinTolerationTime (0.00s)
--- PASS: TestGetMinTolerationTime/no_noExecuteTaints (0.00s)
--- PASS: TestGetMinTolerationTime/no_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/find_the_latest_trigger_time (0.00s)
--- PASS: TestGetMinTolerationTime/trigger_time_is_up (0.00s)
=== FAIL: command-line-arguments TestGetMinTolerationTime/no_noExecuteTaints (unknown)
-1ns=== RUN TestGetMinTolerationTime/no_usedTolerations
0s=== RUN TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations
59.999932917s=== RUN TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil
-1ns=== RUN TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil
-1ns=== RUN TestGetMinTolerationTime/find_the_latest_trigger_time
59.999909167s=== RUN TestGetMinTolerationTime/trigger_time_is_up
0s--- PASS: TestGetMinTolerationTime (0.00s)
--- PASS: TestGetMinTolerationTime/no_noExecuteTaints (0.00s)
--- PASS: TestGetMinTolerationTime/no_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/find_the_latest_trigger_time (0.00s)
--- PASS: TestGetMinTolerationTime/trigger_time_is_up (0.00s)
DONE 2 tests, 2 failures in 3.979s
```
gotestsum printed FAIL, while rerun using `go test`
```console
$ go test -v -run=TestGetMinTolerationTime pkg/util/helper/*.go
=== RUN TestGetMinTolerationTime
=== RUN TestGetMinTolerationTime/no_noExecuteTaints
-1ns=== RUN TestGetMinTolerationTime/no_usedTolerations
0s=== RUN TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations
59.999899792s=== RUN TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil
-1ns=== RUN TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil
-1ns=== RUN TestGetMinTolerationTime/find_the_latest_trigger_time
59.999854833s=== RUN TestGetMinTolerationTime/trigger_time_is_up
0s--- PASS: TestGetMinTolerationTime (0.00s)
--- PASS: TestGetMinTolerationTime/no_noExecuteTaints (0.00s)
--- PASS: TestGetMinTolerationTime/no_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/with_noExecuteTaints_and_usedTolerations (0.00s)
--- PASS: TestGetMinTolerationTime/usedTolerantion.TolerationSeconds_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/noExecuteTaints.TimeAdded_is_nil (0.00s)
--- PASS: TestGetMinTolerationTime/find_the_latest_trigger_time (0.00s)
--- PASS: TestGetMinTolerationTime/trigger_time_is_up (0.00s)
PASS
ok command-line-arguments 0.477s
```
Is this something wrong with gotestsum or did I miss setting any flags?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the reported `gotestsum -- -v -run=TestGetMinTolerationTime pkg/util/helper/*.go` command and compare its output with `go test`. Trace how gotestsum parses the test output and builds the summary; done means the passing test and subtests no longer appear as failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100