vitest-dev / vitest-dev/vitest

Test sequence causes tests to consistently fail or pass

Open
#9,499 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

pending triage
Dominant language
TypeScript
Stars
17.1k
Forks
2k
Avg merge
1d 22h
Merged PRs (30d)
94

Description

Describe the bug

I am experiencing flaky tests. I've tracked it down to it something having to do with mocks/spies in combination with two specific tests being executed in a specific order by 1 worker/thread/fork/thing.

We (ideally) don't use test isolation, but from what I understand that's only if you start messing with global stuff (eg fetch, process.env) anyway. I've also tried various options of clearMocks, mockReset and restoreMocks, and it seems to trigger in all three unless I enable test isolation.

I understand that disabling test isolation might cause tests to have side effects that leak over to other tests, but I've narrowed it down to a reproducable example where it's just two very simple test files which if ordered alphabetically consistently fail. And I don't understand why. It also feels like this behaviour has gotten more common since vitest 4.

it's possible that this isn't technically a bug, or maybe our setup is just wrong, but I can't find anything about this in the documentation nor existing issues. But explicitly mocking a file and it still somehow calling the real thing feels suspicious, regardless of the test order.

Reproduction

https://github.com/barbeque-squared/vitest-mockbug

npm ci
npm run test # will fail
npm run test:pass # will pass, the only difference is the test order
System Info
System:
    OS: Linux 6.18 Arch Linux
    CPU: (12) x64 13th Gen Intel(R) Core(TM) i5-1335U
    Memory: 5.59 GB / 15.28 GB
    Container: Yes
    Shell: 5.3.9 - /usr/bin/bash
  Binaries:
    Node: 25.3.0 - /usr/bin/node
    npm: 11.7.0 - /usr/bin/npm
    pnpm: 10.28.0 - /usr/bin/pnpm
  Browsers:
    Chromium: 144.0.7559.59
    Firefox: 147.0.1
    Firefox Developer Edition: 147.0.1
  npmPackages:
    vitest: ^4.0.17 => 4.0.17
Used Package Manager

npm

Validations

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked minimal reproduction: run npm ci, then npm run test and npm run test:pass to compare the two test orders. Trace why the explicitly mocked module can call the real implementation without test isolation, and consider the issue done when the order-dependent behavior is explained and a regression test or fix is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.