gotestyourself / gotestyourself/gotestsum

Go 1.27: detect failure message based on OutputType

Open
#571 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.7k
Forks
171
PR merge metrics
No merged PRs in 30d

Description

https://go.dev/doc/go1.27#go-test:

> go test -json now annotates "Action":"output" lines with an optional new field "OutputType", specifying the type of output. Currently, the possible values include “error”, “error-continue”, and “frame”. See [cmd/test2json help](https://go.dev/cmd/test2json#hdr-Output_Format) for details.

This could be used by gotestsum to treat the error log output differently:
- In JUnit XML, put only the error log into the failure message. The full output then goes to the stdout (including the failure message).
- In output formats that support it, highlight the error log (e.g. via color).

I'm primarily interested in the JUnit XML aspect. In Kubernetes, we have `go test` based integration tests which produce a lot of log output in addition to the actual failure. I added post-processing of the JUnit file to filter those log lines which can be recognized as normal log output out of the failure message (deny list approach). The OutputType would allow inverting the logic.

There are some caveats which imply that relying on OutputType for JUnit should be opt-in:
- The full log might be expected in the failure message.
- A test binary might not have been compiled with Go 1.27.
- A test author might use `t.Log("failure"); t.Fail()`. This can be mitigated a bit by keeping the full output as failure message if a failed test has no log output with `OutputType: error`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how gotestsum consumes `go test -json` Action output and produces JUnit XML. Review the OutputType compatibility caveats in the issue, then verify that any opt-in behavior distinguishes error output from full test logs while preserving existing behavior for older test binaries and tests without error output.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.