vitest-dev / vitest-dev/vitest
Consider adopting the Open Test Reporting format
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 17.1k
- Forks
- 2k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 94
Description
Clear and concise description of the problem
TL;DR The JUnit team has defined a new language-agnostic test reporting format and implemented a CLI tool for validation, conversion, and HTML report generation. We're reaching out to well-known testing frameworks and reporting tools to ask for feedback and, ultimately, adoption, if you think this format provides value to your users.
Vitest provides a reporter for the "JUnit XML" format. This format did not originate from the JUnit project but was initially introduced by the Ant build tool and then adopted by other Java build tools like Maven and Gradle. Many build servers know how to parse the XML-based format, and even non-Java tools often support it. However, it’s based on the concept of test classes and methods, so using it for frameworks and tools where those elements are not present is awkward at best. Moreover, it does not support nested structures beyond a simple parent-child relationship. Finally, it is not extensible: no additional attributes can be added without the risk of breaking existing tools.
Suggested solution
To overcome these limitations, the JUnit team is defining a new format for test reporting. Its goal is to be platform-agnostic so that as many testing frameworks as possible can benefit from it. Moreover, it is designed to be extensible so new data can be added as needed, without breaking consumers. However, all well-known attributes are properly defined so it remains consumable by downstream reporting tools.
Adding a reporter supporting this new format would allow including more information and allow creating a unified report for frontend and backend tests using the Open Test Reporting CLI.
Benefits
- Easy to write event-based XML format
- Supports infrastructure information, Git metadata, tags, data/file attachments
- Custom data can be added by defining an extension schema
- Language-agnostic
- No Java-specific elements in the core schema
- Not tied to the concept of test classes and methods
- Full support for nested structures
- Extensible HTML report generator
Alternative
No response
Additional context
The JUnit team would be happy to get your feedback on this initiative. We can discuss here or you're welcome to start a thread or open an issue in the Open Test Reporting repo. Should you consider adopting the new format, we'd be happy to provide guidance but we won't have the resources to actually contribute an implementation.
Validations
- Follow our Code of Conduct
- Read the Contributing Guidelines.
- Read the docs.
- Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no Vitest files or tests. Start by reviewing the existing JUnit XML reporter and the Open Test Reporting format and CLI linked in the issue; done would require an agreed adoption scope and implementation plan, since no concrete acceptance criteria are provided.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100