Two tests fail under full-tree parallel load and pass in isolation: engine progress query and the LSP change-storm settle

Recording two load-sensitive failures observed in one session, each once, neither reproducible in isolation. Filed because the evidence is concrete and the next person to see one should not have to re-derive it — not because either is diagnosed.

Problem or observed behavior

Running go test -short ./pkg/flowstate/v1/... over the whole subtree (many packages in parallel) produced two different failures across six runs. Both tests pass repeatedly when run alone. Both are tests whose subject is a race, which is what makes them worth recording rather than dismissing: a test that asserts about settling is the kind that fails for a real reason.

1. engine.TestProgressIsAnsweredWhileTheRunIsStillSettingUp — pkg/flowstate/v1/engine/progress_test.go:115

Received unexpected error:
unknown queryType flowstate.progress. KnownQueryTypes=[__stack_trace __open_sessions __temporal_workflow_metadata]
Messages: a query in the first moments of a run was refused, which reads as a broken worker

The query arrived before flowstate.progress was registered. The test asks at askDuring(t, env, time.Millisecond) — a 1ms delayed callback racing the registration the test exists to prove happens first. The handler really is registered before the vars activity (which is the property under test); what is fragile is the 1ms, which is a wall-clock guess about how far the workflow goroutine gets before the callback fires.

2. lsp.TestHoverThroughAChangeStormAnswersFromTheLatestVersion — pkg/flowstate/v1/flowfile/lsp/requestrace_test.go:217

Error: Not equal:
expected: "...id: final_marker..."
actual  : "...id: shout..."
Messages: the document did not settle on the newest text

This one is more interesting, because it sits inside a testing/synctest bubble and the assertion follows synctest.Wait(). shout is the unmodified raceSource — not one of the eight versions the test sends, but the text before any of them. A bubbled assertion after Wait() should be deterministic, so either something the LSP server does during a change burst is not contained by the bubble (real I/O, a goroutine started outside it, a non-bubbled channel), or the document store can discard a burst entirely under scheduling pressure.

Evidence

• Six full-subtree runs: failure 1 occurred once, failure 2 once, four clean.
• Failure 2 occurred on an unmodified tree at 0a1e780, so neither is caused by the change being tested at the time.
• Non-reproduction: -count=30 on the LSP test in isolation, green; 5 isolated runs and 3 full-package runs of engine, green; two further full-subtree runs after the first two failures, green.
• Both failures are in tests whose own comments describe the race they pin — progress_test.go:92-100 and requestrace_test.go:190-194.

Desired outcome

Each failure is either explained and made deterministic, or shown to be a test-harness artifact and made robust. For the LSP one specifically: whether the bubble actually contains everything a change burst touches is a question worth answering independently of this flake, because the answer decides whether that test proves what it claims.

Acceptance criteria

• TestProgressIsAnsweredWhileTheRunIsStillSettingUp no longer depends on a wall-clock guess for when the query lands, or it is documented why 1ms is sound.
• TestHoverThroughAChangeStormAnswersFromTheLatestVersion either becomes deterministic under load, or the bubble-containment gap that lets it observe pre-burst text is identified.
• Whatever is found, a run of the full subtree under load repeats cleanly enough to justify the claim.

Constraints and dependencies

Neither failure blocks anything today: both are test-side, CI has not shown them, and both packages are green in the ordinary per-package runs CI does. The LSP one touches a testing/synctest user, so anything learned there is worth reflecting in the guidance tools/wallclock's doc gives about what a bubble does and does not contain.

Open questions

• Does the LSP document store's build path start any goroutine outside the bubble, or perform real I/O? That is the first thing to check, and it decides whether this is a test bug or a containment bug.
• Is 1ms in askDuring load-bearing, or would a RegisterDelayedCallback at 0 be equivalent given Temporal's test-env time skipping?

Generated by Claude Code

Start with pkg/flowstate/v1/engine/progress_test.go:92-115 and pkg/flowstate/v1/flowfile/lsp/requestrace_test.go:190-217, then run the cited tests and the full subtree command under load. Trace the delayed query, change burst, document-store build path, and testing/synctest bubble boundaries, including tools/wallclock guidance. Done means both failures are explained and deterministic or robust, with repeated full-subtree runs clean.