vitest-dev / vitest-dev/vitest
Feature request: Make "--changed" respect coverage or use static code analysis
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 17.1k
- Forks
- 2k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 94
Description
Clear and concise description of the problem
What if Vitest kept track of which lines each test runs through. That way, the "--changed" flag would be truly only "changed", and could save a huge amount of test execution time.
In our test suite, we have a lot of end-to-end tests, as well as a lot of integration tests. So here, we don't care about Vitest's performance, because that's typically not the limiting factor. IO such as database operations and API calls is limiting instead.
So for that, to really cut off some test execution time, we need to be able to better restrict what tests are run.
--affected does a great job already by looking at the import tree, but it could become even better. If this was implemented, it would be equal to how it works in IntelliJ based IDEs like WebStorm, IntelliJ and Rider, and it could be one of the only test libraries for node that would support this. So there's a lot of opportunity here.
Suggested solution
The way I see it, it could be achieved in three ways.
- Using code coverage: Some tools already do this today. However, it may require generating coverage in a specific format which includes this information. Perhaps there's also some performance considerations in basing it on files (so maybe suggestion 2 is better).
- Using in-memory code coverage: Same as above, but maybe based on some kind of in-memory data. Vitest must generate existing coverage files today based on some data in memory (that's an assumption), so this could maybe be "easy" to implement directly into the "watch" mechanism of Vite.
- Analyzing the AST: Similarly to how imports are analyzed today, we could use the Visitor design pattern or something similar to be able to recursively traverse function calls etc. But that might be more complex, as Node projects can be configured in a variety of ways.
Alternative
No response
Additional context
This was sparked here: https://github.com/vitest-dev/vitest/discussions/6734
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the existing --affected behavior and the watch mechanism referenced in the issue, then read discussion #6734 for the motivating context. Compare the proposed coverage, in-memory coverage, and AST approaches; the issue does not define a chosen approach or concrete completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100