gotestyourself / gotestyourself/gotestsum

Go subtests reported twice

Open
#203 2 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
2.7k
Forks
171
PR merge metrics
No merged PRs in 30d

Description

When a subtest fails, gotestsum reports both the test and the subtest:

```
$ gotestsum --format=standard-quiet ./...
FAIL
FAIL example.com/greetings 0.125s

=== Failed
=== FAIL: . TestSayHello/no_input (0.00s)
greetings_test.go:20: want: Hello Person, got: Hello
--- FAIL: TestSayHello/no_input (0.00s)

=== FAIL: . TestSayHello (0.00s)

DONE 4 tests, 2 failures in 0.389s
```

The same test failure run with plain `go test` also shows the nesting but it is only reported once:
```
$ go test ./...
--- FAIL: TestSayHello (0.00s)
--- FAIL: TestSayHello/no_input (0.00s)
greetings_test.go:20: want: Hello Person, got: Hello
FAIL
FAIL example.com/greetings 0.116s
FAIL
```

Is there a way around that? This shows up as two separate test failures in CircleCI which impacts our ability to accurately report on test failures without having to manually dedupe the data. The output in the CircleCI UI looks like this:

CircleCI Subtests

Here's the JSON output from `go test --json` which seems to match the gotestsum JSON output as well:

```
{"Time":"2021-06-30T16:03:22.288363Z","Action":"run","Package":"example.com/greetings","Test":"TestSayHello"}
{"Time":"2021-06-30T16:03:22.288528Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello","Output":"=== RUN TestSayHello\n"}
{"Time":"2021-06-30T16:03:22.288537Z","Action":"run","Package":"example.com/greetings","Test":"TestSayHello/no_input"}
{"Time":"2021-06-30T16:03:22.288543Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/no_input","Output":"=== RUN TestSayHello/no_input\n"}
{"Time":"2021-06-30T16:03:22.28858Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/no_input","Output":" greetings_test.go:20: want: Hello Person, got: Hello \n"}
{"Time":"2021-06-30T16:03:22.288589Z","Action":"run","Package":"example.com/greetings","Test":"TestSayHello/first_name"}
{"Time":"2021-06-30T16:03:22.288593Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/first_name","Output":"=== RUN TestSayHello/first_name\n"}
{"Time":"2021-06-30T16:03:22.288597Z","Action":"run","Package":"example.com/greetings","Test":"TestSayHello/full_name"}
{"Time":"2021-06-30T16:03:22.2886Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/full_name","Output":"=== RUN TestSayHello/full_name\n"}
{"Time":"2021-06-30T16:03:22.288608Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello","Output":"--- FAIL: TestSayHello (0.00s)\n"}
{"Time":"2021-06-30T16:03:22.288612Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/no_input","Output":" --- FAIL: TestSayHello/no_input (0.00s)\n"}
{"Time":"2021-06-30T16:03:22.288616Z","Action":"fail","Package":"example.com/greetings","Test":"TestSayHello/no_input","Elapsed":0}
{"Time":"2021-06-30T16:03:22.288625Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/first_name","Output":" --- PASS: TestSayHello/first_name (0.00s)\n"}
{"Time":"2021-06-30T16:03:22.288727Z","Action":"pass","Package":"example.com/greetings","Test":"TestSayHello/first_name","Elapsed":0}
{"Time":"2021-06-30T16:03:22.288736Z","Action":"output","Package":"example.com/greetings","Test":"TestSayHello/full_name","Output":" --- PASS: TestSayHello/full_name (0.00s)\n"}
{"Time":"2021-06-30T16:03:22.28874Z","Action":"pass","Package":"example.com/greetings","Test":"TestSayHello/full_name","Elapsed":0}
{"Time":"2021-06-30T16:03:22.288743Z","Action":"fail","Package":"example.com/greetings","Test":"TestSayHello","Elapsed":0}
{"Time":"2021-06-30T16:03:22.288746Z","Action":"output","Package":"example.com/greetings","Output":"FAIL\n"}
{"Time":"2021-06-30T16:03:22.288888Z","Action":"output","Package":"example.com/greetings","Output":"FAIL\texample.com/greetings\t0.102s\n"}
{"Time":"2021-06-30T16:03:22.288903Z","Action":"fail","Package":"example.com/greetings","Elapsed":0.102}
```

Here is the example function:

```go
package greetings

func SayHello(s string) string {
return "Hello " + s
}

```

And its test:

```go
package greetings

import "testing"

func TestSayHello(t *testing.T) {
tests := []struct {
name string
input string
output string
}{
{"no input", "", "Hello Person"},
{"first name", "First", "Hello First"},
{"full name", "First Last", "Hello First Last"},
}

for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
got := SayHello(test.input)
if test.output != got {
t.Fatalf("want: %s, got: %s", test.output, got)
}
})
}
}

```

The `go.mod` file is simply:

```
module example.com/greetings

go 1.15

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the duplicate reporting with the provided greetings_test.go example, then trace how gotestsum consumes the go test --json event stream and formats its JSON output. Compare parent and subtest fail events; done means a single failed test is reported for the nested failure while the existing output remains accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.