bazel-contrib / bazel-contrib/rules_go

wrong xml test report if panic from TestMain

Open
#3,152 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

### What version of rules_go are you using?
v0.31.0

### What version of gazelle are you using?
v0.25.0

### What version of Bazel are you using?
5.1.1

### Does this issue reproduce with the latest releases of all the above?

yes

### What operating system and processor architecture are you using?

darwin, amd64

### Any other potentially useful information about your toolchain?

### What did you do?

I added the following test to the package `./manual` and execute `bazel test //manual/...`
```
package manual_test

import "testing"

func TestMain(m *testing.M) {
if true {
panic("main")
}
m.Run()
}

func TestManual(t *testing.T) {
}
```

### What did you expect to see?

I expect test to fail and to return the following test report with set execution time
```

```

### What did you see instead?
the execution time is empty
```

```

### dig into the issue

I debugged `go/tools/bzltestutil/wrap.go` and `go/tools/builders/generate_test_main.go` and it seems like the `testing.MainStart` does generate less output comparing to `go test ./manual`, and this is why wrapper can not fill in the actual test execution time.

```
➜ go test ./manual/
panic: main

goroutine 1 [running]:
github.com/bazelbuild/rules_go/manual_test.TestMain(...)
/Users/dududko/Projects/src/github.com/bazelbuild/rules_go/manual/manual_test.go:7
main.main()
_testmain.go:45 +0x12e
FAIL github.com/bazelbuild/rules_go/go/tools/bzltestutil/manual 0.053s
FAIL
```

```
➜ bazel run //manual/...
Executing tests from //go/tools/bzltestutil/manual:bzltestutil_test
-----------------------------------------------------------------------------
panic: main

goroutine 1 [running]:
manual/bzltestutil_test_test.TestMain(...)
manual/manual_test.go:7
main.main()
bazel-out/darwin-fastbuild/bin/manual/bzltestutil_test_/testmain.go:98 +0x179
```

If `go test` prints to stdout `FAIL` messages, then the `testing.MainStart` does not print anything. Altho it does print the correct output when test pass or when panic is raised from inside the test case.

This is the xml test reports does have an empty time field.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the panic in package ./manual with `bazel test //manual/...`. Read go/tools/bzltestutil/wrap.go and go/tools/builders/generate_test_main.go, comparing their behavior with the reported `go test` output. Done means a panic from TestMain produces an XML report whose testsuite time field contains the execution time rather than an empty value.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.