vitest-dev / vitest-dev/vitest

JSON reporter doesn't report unhandled exceptions

Open
#8,669 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feat: reporters
Dominant language
TypeScript
Stars
17.1k
Forks
2k
Avg merge
1d 22h
Merged PRs (30d)
94

Description

Describe the bug

The JSON reporter (and AFAIK most other non-default reporters) don't report unhandled exceptions. They do exit with a non-zero exit code, though.

In particular for the JSON reporter:

  • The value of the root success field in the output JSON is true when it should be false.
  • The exception details are discarded, which is problematic for CI setups that rely on JSON.

Notably, the default reporter prints out a message that includes unhandled exceptions, like:

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: This error might be unhandled!
 ❯ eval example.test.ts:6:11
      4|   (async () => {
      5|     await new Promise((resolve) => setTimeout(resolve, 100));
      6|     throw new Error('This error might be unhandled!');
       |           ^
      7|   })();
      8|   await new Promise((resolve) => setTimeout(resolve, 1000));

This error originated in "example.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
The latest test that might've caused the error is "probably doesn't handle async errors well". It might mean one of the following:
- The error was thrown, while Vitest was running this test.
- If the error occurred after the test had been completed, this was the last documented test before it was thrown.
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

I would expect the JSON reporter to similarly list any unhandled exceptions it has captured in one of the JSON fields, since it did have a non-zero exit code.

Note: Looks like the corresponding field in jest would be testExecError: https://github.com/jestjs/jest/blob/main/website/versioned_docs/version-30.0/Configuration.md#testresultsprocessor-string
However, I think it's possible for vitest to report multiple errors so we might need a new field for vitest.

Reproduction

All that's required is the JSON reporter and any test resulting in an unhandled exception, such as a promise that throws but is not directly awaited in the test, e.g.:

import { it } from 'vitest';
it("probably doesn't handle async errors well", async () => {
  console.log('Hello, World!');
  (async () => {
    await new Promise((resolve) => setTimeout(resolve, 100));
    throw new Error('This error might be unhandled!');
  })();
  await new Promise((resolve) => setTimeout(resolve, 1000));
});

https://stackblitz.com/edit/vitest-dev-vitest-wauyqkhs?file=example.test.ts&view=editor

Tested against 3.2.4 - the latest version at time of discovery.

System Info
System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 20.19.1 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.8.2 - /usr/local/bin/npm
    pnpm: 8.15.6 - /usr/local/bin/pnpm
  npmPackages:
    vitest: 3.2.4 => 3.2.4
Used Package Manager

npm

Validations

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the JSON reporter entry point and reproduce the issue using the example.test.ts case, where an unhandled asynchronous error is thrown. Compare its output with the default reporter and verify that the JSON result marks success as false and preserves the captured exception details.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.