vitest-dev / vitest-dev/vscode

Failing test.fails shows as failed instead of passed

Open
#812 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

pending triage
Dominant language
TypeScript
Stars
924
Forks
122
Avg merge
1d 14h
Merged PRs (30d)
6

Description

Describe the bug

The test case below is expected to fail:

test.fails('expected to fail', async () => {
    await new Promise((resolve) => setTimeout(resolve, 200))
    expect(1 + 1).toBe(3)
})

However, running the test using the VS Code extension shows a red cross. Running the test using the vitest CLI correctly shows a green check mark.

The timeout is needed to reproduce the error, and it needs to be long enough to consistently reproduce it which I suspect is related to how vitest throttles updates.

I also tried to take a quick look at how VS Code handles test results and I think (?) this code that ignores requests to update the state of a test from failed to passed could be relevant.

Expected behavior

I expect a green check mark to appear for failing tests that are marked with fails, not a red cross. This would be consistent with how the vitest CLI displays results for tests marked with fails, see below:

> vitest
 ✓ test/basic.test.ts (1 test) 208ms
   ✓ expected to fail 207ms

 Test Files  1 passed (1)
      Tests  1 expected fail (1)
   Start at  10:06:38
   Duration  296ms (transform 7ms, setup 0ms, import 12ms, tests 208ms, environment 0ms)
Reproduction

https://github.com/danieladugyan/vitest-repro

Output
[INFO 9:55:17 AM] [v1.50.8] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[INFO 9:55:17 AM] [API] Resolving configs: vitest-repro/vite.config.ts
[9:55:17 AM] [API] Spawning on-demand process...
[INFO 9:55:17 AM] [API] Running Vitest v4.1.11 (vitest-repro/vite.config.ts) with "/Users/user/.nvm/versions/node/v24.20.0/bin/node /Users/user/.vscode/extensions/vitest.explorer-1.50.8/dist/worker.js"
[9:55:17 AM] [API] Vitest WebSocket connection closed, cannot call RPC anymore.
[9:55:17 AM] [API] On-demand process finished in 527ms
[INFO 9:55:17 AM] [API] Watching vite.config.ts
[INFO 9:55:17 AM] [VSCODE] Watching vitest-repro with pattern **/*
[INFO 9:55:18 AM] [API] Collecting tests: test/basic.test.ts
[9:55:18 AM] [API] Spawning on-demand process...
[INFO 9:55:18 AM] [API] Running Vitest v4.1.11 (vitest-repro/vite.config.ts) with "/Users/user/.nvm/versions/node/v24.20.0/bin/node /Users/user/.vscode/extensions/vitest.explorer-1.50.8/dist/worker.js"
[9:55:18 AM] [API] Vitest WebSocket connection closed, cannot call RPC anymore.
[9:55:18 AM] [API] On-demand process finished in 169ms
[INFO 9:55:21 AM] [API] Running Vitest v4.1.11 (vitest-repro/vite.config.ts) with "/Users/user/.nvm/versions/node/v24.20.0/bin/node /Users/user/.vscode/extensions/vitest.explorer-1.50.8/dist/worker.js"
[9:55:21 AM] Enqueuing "expected to fail"
[INFO 9:55:21 AM] Running 1 file(s) with name pattern: ^\s?expected to fail$
[INFO 9:55:21 AM] [DECOR] Reset all decorations.
[9:55:21 AM] No task result for "basic.test.ts", ignoring
[9:55:21 AM] Enqueuing "expected to fail"
[9:55:21 AM] No errors found for "basic.test.ts"
[9:55:21 AM] Marking "expected to fail" as running (state is run)
[9:55:21 AM] Marking "expected to fail" as failed with 1 errors
[9:55:21 AM] Marking "expected to fail" as passed
[9:55:21 AM] No errors found for "basic.test.ts"
[9:55:21 AM] Ending test run test/basic.test.ts
[9:55:21 AM] Disposing test runner
[9:55:21 AM] [API] Vitest WebSocket connection closed, cannot call RPC anymore.
Extension Version

1.50.8

Vitest Version

4.1.11

Validations

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction in test/basic.test.ts and trace the result updates through the extension, especially the Vitest runner reference in packages/vitest/src/runtime/runner/run.ts and VS Code's testResult.ts. Reproduce the delayed test.fails case and verify that the extension ends with a green check mark, matching the Vitest CLI result.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript, vscode
Domain
developer-experience, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.