BHoM / BHoM/Test_Toolkit

Improve NUnit result plotting

Offen
#421 0 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @alelom Auf GitHub ansehen
type:feature
Vorherrschende Sprache
C#
Sterne
0
Forks
1
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

#### Description:

Currently, the NUnit results when there is a failure are plotted like this (example taken [from here](https://github.com/BHoM/BHoM_Engine/blob/45d9d73dd52afcad196ad1f1db100492b4b1d99b/.ci/unit-tests/Base_Engine_Tests/Query/Unpack.cs)):

![image](https://user-images.githubusercontent.com/6352844/220681404-39f74861-3328-4faf-bdee-1e5927388a20.png)

The test failing here is only one (`BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest`), but there are other 2 tests that were run. Additionally, those two test may have some RecordMessage log that could be useful to show (see #420).

After discussing with @FraserGreenroyd , we agreed on this example structure for the results plotting:
```
Inner results:
BH.Tests.Engine.Base.Query.UnpackTests.ErrorReportingTest (Passed):
Messages:
Some error logged via BH.Engine.Base.Compute.RecordError
Some error logged via BH.Engine.Base.Compute.RecordWarning
Some error logged via BH.Engine.Base.Compute.RecordNote

BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest (Failed):
Shouldly.ShouldAssertException : Comparing object equivalence, at path:
result.OfType().FirstOrDefault() [BH.oM.Base.BHoMObject]
BHoM_Guid [System.Guid]
Expected value to be
a1380fbb-b004-4bac-b28c-0d45780a5e9c
but was
4fa87007-d24a-d44b-9d3e-aa9d743b7059
at BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest() in C:\ProgramData\BHoMGitHub\BHoM_Engine\.ci\unit-tests\Base_Engine_Tests\Query\Unpack.cs:line 143

Error:
Some warning logged via BH.Engine.Base.Compute.RecordError

Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning

BH.Tests.Engine.Base.Query.UnpackTests.WarningReportingTest (Passed):
Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning

BH.Tests.Engine.Base.Query.UnpackTests.NoteReportingTest (Passed):
Messages:
Some note logged via BH.Engine.Base.Compute.RecordNote
```

The information from the RecordMessage log is shown differently depending on whether the test has passed or failed:
- when the test has passed, the recorded messages are clustered under a "Messages" category (no differentiation btw Errors/Warnings/Note). This was suggested by @FraserGreenroyd to avoid confusing readers.
- when the test has failed, the recorded messages are grouped as normal (Errors/Warnings/Notes).

For each test, the recorded messages should appear after any message from NUnit.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.