bazel-contrib / bazel-contrib/rules_go
wrong xml test report if panic from TestMain
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 760
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 12
Description
### What version of rules_go are you using?
v0.31.0
### What version of gazelle are you using?
v0.25.0
### What version of Bazel are you using?
5.1.1
### Does this issue reproduce with the latest releases of all the above?
yes
### What operating system and processor architecture are you using?
darwin, amd64
### Any other potentially useful information about your toolchain?
### What did you do?
I added the following test to the package `./manual` and execute `bazel test //manual/...`
```
package manual_test
import "testing"
func TestMain(m *testing.M) {
if true {
panic("main")
}
m.Run()
}
func TestManual(t *testing.T) {
}
```
### What did you expect to see?
I expect test to fail and to return the following test report with set execution time
```
```
### What did you see instead?
the execution time is empty
```
```
### dig into the issue
I debugged `go/tools/bzltestutil/wrap.go` and `go/tools/builders/generate_test_main.go` and it seems like the `testing.MainStart` does generate less output comparing to `go test ./manual`, and this is why wrapper can not fill in the actual test execution time.
```
➜ go test ./manual/
panic: main
goroutine 1 [running]:
github.com/bazelbuild/rules_go/manual_test.TestMain(...)
/Users/dududko/Projects/src/github.com/bazelbuild/rules_go/manual/manual_test.go:7
main.main()
_testmain.go:45 +0x12e
FAIL github.com/bazelbuild/rules_go/go/tools/bzltestutil/manual 0.053s
FAIL
```
```
➜ bazel run //manual/...
Executing tests from //go/tools/bzltestutil/manual:bzltestutil_test
-----------------------------------------------------------------------------
panic: main
goroutine 1 [running]:
manual/bzltestutil_test_test.TestMain(...)
manual/manual_test.go:7
main.main()
bazel-out/darwin-fastbuild/bin/manual/bzltestutil_test_/testmain.go:98 +0x179
```
If `go test` prints to stdout `FAIL` messages, then the `testing.MainStart` does not print anything. Altho it does print the correct output when test pass or when panic is raised from inside the test case.
This is the xml test reports does have an empty time field.
Contributor guide
Research direction
Start by reproducing the panic in package ./manual with `bazel test //manual/...`. Read go/tools/bzltestutil/wrap.go and go/tools/builders/generate_test_main.go, comparing their behavior with the reported `go test` output. Done means a panic from TestMain produces an XML report whose testsuite time field contains the execution time rather than an empty value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100