Nimblesite / Nimblesite/SharpLsp
Flaky e2e: stale F# error diagnostic survives 60s after file correction (clears-diagnostics race)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 132
- Forks
- 5
- Avg merge
- 6h 24m
- Merged PRs (30d)
- 27
Description
Symptom
Flaky e2e failure on CI (observed on PR #141, run 29000528962, attempt 1 — 535/536 tests passed):
F# LSP — Diagnostics › clears diagnostics when the file is corrected
AssertionError [ERR_ASSERTION]: corrected file must clear its error diagnostics
1 !== 0
at Context.<anonymous> (out/test/suite/fsharp-lsp-diagnostics.test.js:96:16)
Why this is likely NOT a test-side timing problem
The test (editors/vscode/src/test/suite/fsharp-lsp-diagnostics.test.ts:65) already polls
vscode.languages.getDiagnostics(uri) every 2s for up to 60s (DIAG_TIMEOUT_MS) after
the corrected content is written to disk and the fixture is re-opened. A stale FCS error
diagnostic surviving a full 60 seconds after correct content was on disk + re-opened points
at a product-side race, not a too-tight wait.
Hypothesis
Race in the pull-diagnostics pipeline around close/re-open:
- Broken content analyzed by FCS sidecar (slow — cold or under CI load).
- Test restores correct content on disk,
closeAllEditors()firesdidClose, re-open firesdidOpenwith corrected text. - The in-flight analysis of the stale content completes after the re-open, and its
result is published/cached against the document with no newer pull invalidating it —
stale diagnostics persist indefinitely (until another edit forces a fresh pull).
Relevant components: src/diagnostics.rs, src/pull_diagnostics.rs (result-ID/versioning of
pull results vs. document lifecycle events), F# sidecar FSharpChecker invalidation.
If confirmed, this is user-visible: correcting a file can leave phantom error squiggles.
Sequence in the failing test
- write broken
DiagnosticsTarget.fs→ open → poll until FCS error appears ✓ - restore valid content → close all editors → re-open → poll 60s for errors to clear ✗ (1 error remained)
Next steps
- Reproduce locally under load (run the vsix suite with CPU pressure; loop this single test)
- Add version/result-ID guard so a pull response for an older document version can never overwrite a newer one (bug-fix process: failing test first — the e2e test above already covers it; consider a targeted host-side test for the version race)
- Verify FCS
InvalidateFile/checker cache behavior on didClose→didOpen of the same path
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with editors/vscode/src/test/suite/fsharp-lsp-diagnostics.test.ts:65 and reproduce the single test under load to confirm the race. Then inspect src/diagnostics.rs and src/pull_diagnostics.rs, including FSharpChecker invalidation around didClose and didOpen. Done means stale pull results cannot overwrite diagnostics for newer document content and the e2e test clears the error reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fsharp, rust, typescript, vscode
- Domain
- devtools, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100