gate: a go.mod diff that changes no selected version still forces the whole-module race leg, and the leg reveals nothing until it ends
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
Filed at #1961's invitation, where this was recorded under "Not in scope" as a feedback-loop cost worth its own issue. It has a sharper cause than the buffering noted there, and the two compound.
Problem or observed behavior
tools/gate treats any go.mod diff as module-wide, so it runs GOMEMLIMIT=2GiB go test -race -p=1 -timeout 900s ./... — twenty-five minutes and change on this tree, serialized by -p=1. That is correct for a go.mod diff that moves a version, adds a module, or drops one: every package's build inputs really may have changed, and the leg's own comment says so ("when go.mod moved, every package is affected, so there is no narrower answer to give", tools/gate/main.go:349-351).
It is not correct for a go.mod diff that only moves a requirement between the direct and indirect blocks. The selected version is identical, the module graph is identical, and go.sum does not change. The // indirect marker is bookkeeping go mod tidy maintains; it is not an input to compiling anything. So the widest and slowest leg in the gate is forced by a diff that provably cannot change what any package compiles against.
Compounding it, and this is #1961's observation: the leg prints nothing until it finishes, because everything is piped through tools/testsum. A run that is killed, times out, or is read from a truncated log yields no evidence at all — not even the name of the test that failed. The two together mean a contributor whose diff touches go.mod waits ~25 minutes for a single bit, and learns nothing if they cannot wait.
Evidence
From #1991, whose entire go.mod diff is one requirement's block:
$ git diff origin/main...HEAD -- go.mod
- github.com/google/uuid v1.6.0
+ github.com/google/uuid v1.6.0 // indirect
$ git diff origin/main...HEAD -- go.sum | wc -l
0
The gate's own first lines on that diff:
gate: 18 changed file(s) vs merge-base 2048efcce030
gate: vet: running (go.mod changed, so every package is affected)
gate: test: running (go.mod changed, every package is affected)
gate: test: $ GOMEMLIMIT=2GiB go test -json -race -p=1 -timeout 900s ./... | go run ./tools/testsum
That leg ran for over twenty-five minutes on a two-CPU cloud session. Its first run in that session ended:
gate: FAIL test: `GOMEMLIMIT=2GiB go test -json -race -p=1 -timeout 900s ./... | go run ./tools/testsum` failed: go test: exit status 1
gate: 1 leg(s) failed
with no failing test named anywhere in the retained output, because the shell pipeline that captured it kept only the tail and testsum had emitted its summary in one burst. Finding the failure meant re-running packages one at a time — ./pkg/flowstate/v1/server/ (82s), ./pkg/flowstate/v1/plugin/... (57s), both green — which is the loop this issue is about.
That the build list is genuinely unchanged is checkable in one command, which is also the proposed discriminator:
$ go list -m all > /tmp/after && git stash && go list -m all > /tmp/before && git stash pop && diff /tmp/before /tmp/after && echo IDENTICAL
IDENTICAL
Desired outcome
A go.mod diff that changes no selected version and no module in the build list is scoped like any other diff — the affected set — rather than forcing the module. A go.mod diff that does change either still forces the module, unchanged. And whichever leg runs, a failure names the test that failed even when the run is cut short.
The scoping half is the one with the measurement behind it. The output half is the one that makes every future failure cheaper to read, including the ones this issue is not about.
Acceptance criteria
- A diff whose only
go.modchange moves a requirement between the direct and indirect blocks, withgo.sumunchanged, does not select the module-wide test leg; the gate's reason line says which discriminator it applied and what it therefore did not run, in the same voice as the existing residual lines. - A diff that changes a version, adds a module, or removes one still selects the module-wide leg. A regression check covers both directions, since the failing direction here is the one that passes silently by doing more work than needed.
- The discriminator is derived from the toolchain rather than by parsing
go.mod: comparing the build list (go list -m all, orgo mod graph) across the diff is the shape that cannot disagree with whatgo buildactually selects. Parsing the// indirectcomment would be a second representation of a fact the toolchain already answers. - A failing test leg names at least one failing test in output that survives the run being killed — streamed, or written to a file the gate names on failure.
Constraints and dependencies
-p=1 is deliberate and should stay: tools/gate/main.go:882-884 records that packages are serialized so their Temporal processes do not consume one another's wall-clock budgets, and #1980 is two tests that fail under full-tree parallel load and pass in isolation, which is the same pressure seen from the other side. Nothing here proposes relaxing it — narrowing which packages run is the lever, not how they run.
#887 priced the three options for the harness-diff forcing and chose to keep test on the affected set there while vet widens. This is the same trade for the module forcing, and the argument is stronger, because in the reclassification case the wider answer is not merely expensive but provably not more informative.
The vet leg has the same escalation on the same condition. It is cheap enough that it is probably not worth a discriminator of its own; worth deciding rather than assuming, since one predicate would serve both.
Open questions
- Is
go list -m allequality the right predicate, or should it be the narrower "every package's compiled inputs are unchanged" thatgo list -depswould answer? The former is one command and cannot be wrong in the unsafe direction; the latter is more precise and more to maintain. - Should the gate cache the before-side build list rather than computing it against the merge base each run? It is a second
go liston a cold module cache, which is not free, and the gate's value is that it answers fast.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tools/gate/main.go around the existing go.mod explanation at lines 349-351 and the serialized test setting at 882-884; inspect how the gate computes affected packages and how tools/testsum handles output. Compare build lists with go list -m all across the diff. Done means unchanged module selections avoid the whole-module leg, real module changes still widen it, and interrupted failures retain a failing test name.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system, ci-cd, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100