formulahendry / formulahendry/vscode-dotnet-test-explorer

Create a custom DataCollector to gather the data from the test run

Open
#269 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
208
Forks
93
PR merge metrics
No merged PRs in 30d

Description

We regularly have issues with the TRX format, which isn't standardized or stable, and returns all test results at once (instead of streaming them in as the tests finish). Using vstest directly has already been tried in #83, with the disadvantage of introducing a lot of additional complexity (+8k LOC).

However, looking at the docs I think that adding only a custom DataCollector, [as described here](https://github.com/microsoft/vstest-docs/blob/master/docs/analyze.md) that serves exactly our needs can be done in maybe a hundred lines of C# code.

For context:
- The DataCollector API provides some events that are being invoked whenever a test is started/finished/... We just need to subclass it and add some handlers.
- A DataCollector can be added by extending the command line of `dotnet test` slightly. The documentation linked above suggests something like
```
/collect:"MyDataCollector" /testadapterpath:
```

The extension could provide a TCP listener to accept the results. They could directly be written as JSON on the DataCollector side, removing the need for an own parser on our side (while still being type-safe thanks to TypeScript's Interfaces).

All in all, I believe that this approach could increase the reliability of the extension while not adding much complexity. I'm just writing this down here to sort my thoughts on it and to enable @stefanforsberg to insert additional ones :-)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.