Review ultra-verbose `go tool dist test` logs for unexpected skips.
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 431
- Forks
- 44
- Avg merge
- 21h 18m
- Merged PRs (30d)
- 30
Description
Broken off from original issue tracking full coverage vs. upstream: https://github.com/microsoft/go/issues/1
Something we can do to be a little more sure our tests are running properly is to look at the verbose logs, with a modification to make them more verbose and show SKIPs:
Here's what I did to make tests show skips:
diff --git a/src/cmd/dist/test.go b/src/cmd/dist/test.go index 0c8e2c56bc..1248d9145b 100644 --- a/src/cmd/dist/test.go +++ b/src/cmd/dist/test.go @@ -271,7 +271,7 @@ func short() string { // defaults as later arguments in the command line. func (t *tester) goTest() []string { return []string{ - "go", "test", "-short=" + short(), "-count=1", t.tags(), t.runFlag(""), + "go", "test", "-v", "-short=" + short(), "-count=1", t.tags(), t.runFlag(""), } } @@ -350,6 +350,7 @@ func (t *tester) registerStdTest(pkg string, useG3 bool) { } args := []string{ "test", + "-v", "-short=" + short(), t.tags(), t.timeout(timeoutSec), @@ -388,6 +389,7 @@ func (t *tester) registerRaceBenchTest(pkg string) { ranGoBench = true args := []string{ "test", + "-v", "-short=" + short(), "-race", t.timeout(1200), // longer timeout for race with benchmarks~500 results for 'SKIP': https://gist.github.com/dagood/2bc08e37da295c2022b9196572d378a4
Results with 1 line of context (skip reason): https://gist.github.com/dagood/8105bad6aff4803794a4ed42c265a4fc
500 lines isn't small, but it isn't impossible. There are duplicate skip reasons, which helps.
Unfortunately, there is no baseline from upstream to check against. We need to reason out whether each one is a problem.
This is based on a golang-devs thread: https://groups.google.com/g/golang-dev/c/PNzwZXOe7bQ/m/M43Gl9mVDAAJ.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/cmd/dist/test.go, especially tester.goTest, registerStdTest, and registerRaceBenchTest, and inspect the verbose go tool dist test output described in the issue. Review the linked SKIP logs and their one-line context, grouping duplicate reasons and checking each for an unexpected omission. Done means documenting which skips are problematic; the issue notes there is no upstream baseline, so comparison requires reasoning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100