BHoM / BHoM/Test_Toolkit

Improve NUnit result plotting

Open
#421 0 comments 0 reactions 1 assignee Claimed by @alelom View on GitHub
type:feature
Dominant language
C#
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

#### Description:

Currently, the NUnit results when there is a failure are plotted like this (example taken [from here](https://github.com/BHoM/BHoM_Engine/blob/45d9d73dd52afcad196ad1f1db100492b4b1d99b/.ci/unit-tests/Base_Engine_Tests/Query/Unpack.cs)):

![image](https://user-images.githubusercontent.com/6352844/220681404-39f74861-3328-4faf-bdee-1e5927388a20.png)

The test failing here is only one (`BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest`), but there are other 2 tests that were run. Additionally, those two test may have some RecordMessage log that could be useful to show (see #420).

After discussing with @FraserGreenroyd , we agreed on this example structure for the results plotting:
```
Inner results:
BH.Tests.Engine.Base.Query.UnpackTests.ErrorReportingTest (Passed):
Messages:
Some error logged via BH.Engine.Base.Compute.RecordError
Some error logged via BH.Engine.Base.Compute.RecordWarning
Some error logged via BH.Engine.Base.Compute.RecordNote

BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest (Failed):
Shouldly.ShouldAssertException : Comparing object equivalence, at path:
result.OfType().FirstOrDefault() [BH.oM.Base.BHoMObject]
BHoM_Guid [System.Guid]
Expected value to be
a1380fbb-b004-4bac-b28c-0d45780a5e9c
but was
4fa87007-d24a-d44b-9d3e-aa9d743b7059
at BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest() in C:\ProgramData\BHoMGitHub\BHoM_Engine\.ci\unit-tests\Base_Engine_Tests\Query\Unpack.cs:line 143

Error:
Some warning logged via BH.Engine.Base.Compute.RecordError

Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning

BH.Tests.Engine.Base.Query.UnpackTests.WarningReportingTest (Passed):
Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning

BH.Tests.Engine.Base.Query.UnpackTests.NoteReportingTest (Passed):
Messages:
Some note logged via BH.Engine.Base.Compute.RecordNote
```

The information from the RecordMessage log is shown differently depending on whether the test has passed or failed:
- when the test has passed, the recorded messages are clustered under a "Messages" category (no differentiation btw Errors/Warnings/Note). This was suggested by @FraserGreenroyd to avoid confusing readers.
- when the test has failed, the recorded messages are grouped as normal (Errors/Warnings/Notes).

For each test, the recorded messages should appear after any message from NUnit.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.