Nimblesite / Nimblesite/SharpLsp

Flaky e2e: stale F# error diagnostic survives 60s after file correction (clears-diagnostics race)

Open
#142 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

.NET bug
Dominant language
TypeScript
Stars
132
Forks
5
Avg merge
6h 24m
Merged PRs (30d)
27

Description

Symptom

Flaky e2e failure on CI (observed on PR #141, run 29000528962, attempt 1 — 535/536 tests passed):

F# LSP — Diagnostics › clears diagnostics when the file is corrected
AssertionError [ERR_ASSERTION]: corrected file must clear its error diagnostics
1 !== 0
    at Context.<anonymous> (out/test/suite/fsharp-lsp-diagnostics.test.js:96:16)

Why this is likely NOT a test-side timing problem

The test (editors/vscode/src/test/suite/fsharp-lsp-diagnostics.test.ts:65) already polls
vscode.languages.getDiagnostics(uri) every 2s for up to 60s (DIAG_TIMEOUT_MS) after
the corrected content is written to disk and the fixture is re-opened. A stale FCS error
diagnostic surviving a full 60 seconds after correct content was on disk + re-opened points
at a product-side race, not a too-tight wait.

Hypothesis

Race in the pull-diagnostics pipeline around close/re-open:

  1. Broken content analyzed by FCS sidecar (slow — cold or under CI load).
  2. Test restores correct content on disk, closeAllEditors() fires didClose, re-open fires didOpen with corrected text.
  3. The in-flight analysis of the stale content completes after the re-open, and its
    result is published/cached against the document with no newer pull invalidating it —
    stale diagnostics persist indefinitely (until another edit forces a fresh pull).

Relevant components: src/diagnostics.rs, src/pull_diagnostics.rs (result-ID/versioning of
pull results vs. document lifecycle events), F# sidecar FSharpChecker invalidation.

If confirmed, this is user-visible: correcting a file can leave phantom error squiggles.

Sequence in the failing test

  • write broken DiagnosticsTarget.fs → open → poll until FCS error appears ✓
  • restore valid content → close all editors → re-open → poll 60s for errors to clear ✗ (1 error remained)

Next steps

  • Reproduce locally under load (run the vsix suite with CPU pressure; loop this single test)
  • Add version/result-ID guard so a pull response for an older document version can never overwrite a newer one (bug-fix process: failing test first — the e2e test above already covers it; consider a targeted host-side test for the version race)
  • Verify FCS InvalidateFile/checker cache behavior on didClose→didOpen of the same path

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with editors/vscode/src/test/suite/fsharp-lsp-diagnostics.test.ts:65 and reproduce the single test under load to confirm the race. Then inspect src/diagnostics.rs and src/pull_diagnostics.rs, including FSharpChecker invalidation around didClose and didOpen. Done means stale pull results cannot overwrite diagnostics for newer document content and the e2e test clears the error reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
fsharp, rust, typescript, vscode
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.