vitest-dev / vitest-dev/vitest
Test sequence causes tests to consistently fail or pass
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 17.1k
- Forks
- 2k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 94
Description
Describe the bug
I am experiencing flaky tests. I've tracked it down to it something having to do with mocks/spies in combination with two specific tests being executed in a specific order by 1 worker/thread/fork/thing.
We (ideally) don't use test isolation, but from what I understand that's only if you start messing with global stuff (eg fetch, process.env) anyway. I've also tried various options of clearMocks, mockReset and restoreMocks, and it seems to trigger in all three unless I enable test isolation.
I understand that disabling test isolation might cause tests to have side effects that leak over to other tests, but I've narrowed it down to a reproducable example where it's just two very simple test files which if ordered alphabetically consistently fail. And I don't understand why. It also feels like this behaviour has gotten more common since vitest 4.
it's possible that this isn't technically a bug, or maybe our setup is just wrong, but I can't find anything about this in the documentation nor existing issues. But explicitly mocking a file and it still somehow calling the real thing feels suspicious, regardless of the test order.
Reproduction
https://github.com/barbeque-squared/vitest-mockbug
npm ci
npm run test # will fail
npm run test:pass # will pass, the only difference is the test order
System Info
System:
OS: Linux 6.18 Arch Linux
CPU: (12) x64 13th Gen Intel(R) Core(TM) i5-1335U
Memory: 5.59 GB / 15.28 GB
Container: Yes
Shell: 5.3.9 - /usr/bin/bash
Binaries:
Node: 25.3.0 - /usr/bin/node
npm: 11.7.0 - /usr/bin/npm
pnpm: 10.28.0 - /usr/bin/pnpm
Browsers:
Chromium: 144.0.7559.59
Firefox: 147.0.1
Firefox Developer Edition: 147.0.1
npmPackages:
vitest: ^4.0.17 => 4.0.17
Used Package Manager
npm
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
- The provided reproduction is a minimal reproducible example of the bug.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked minimal reproduction: run npm ci, then npm run test and npm run test:pass to compare the two test orders. Trace why the explicitly mocked module can call the real implementation without test isolation, and consider the issue done when the order-dependent behavior is explained and a regression test or fix is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100