UX when there is failing PoisonTests in CI could be improved.
- Dominant language
- No language data
- Stars
- 287
- Forks
- 145
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 10
Description
The following information is all that appears when there is a failing poison test in CI
```
Error message
Assert.Null() Failure: Value is not null
Expected: null
Actual: "\nExpected file '/vmr/test/Microsoft.DotNet.Source"···
Stack trace
[at Microsoft.DotNet.SourceBuild.SmokeTests.BaselineHelper.CompareFiles(String expectedFilePath, String actualFilePath, ITestOutputHelper outputHelper, Boolean warnOnDiffs) in /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/BaselineHelper.cs:line 76](https://dev.azure.com/dnceng/internal/_git/ddda0dd6-918f-48ba-ba8f-730d9d8c9320?path=%2Fvmr%2Ftest%2FMicrosoft.DotNet.SourceBuild.SmokeTests%2FBaselineHelper.cs&version=GBmain&_a=contents&line=76&lineEnd=77&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Microsoft.DotNet.SourceBuild.SmokeTests.BaselineHelper.CompareBaselineContents(String baselineFileName, String actualContents, ITestOutputHelper outputHelper, Boolean warnOnDiffs, String baselineSubDir) in /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/BaselineHelper.cs:line 49](https://dev.azure.com/dnceng/internal/_git/ddda0dd6-918f-48ba-ba8f-730d9d8c9320?path=%2Fvmr%2Ftest%2FMicrosoft.DotNet.SourceBuild.SmokeTests%2FBaselineHelper.cs&version=GBmain&_a=contents&line=49&lineEnd=50&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Microsoft.DotNet.SourceBuild.SmokeTests.PoisonTests.VerifyUsage() in /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/PoisonTests.cs:line 30](https://dev.azure.com/dnceng/internal/_git/ddda0dd6-918f-48ba-ba8f-730d9d8c9320?path=%2Fvmr%2Ftest%2FMicrosoft.DotNet.SourceBuild.SmokeTests%2FPoisonTests.cs&version=GBmain&_a=contents&line=30&lineEnd=31&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
```
Furthermore the attached console output only includes the following:
```
Executing: git diff --no-index /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/assets/baselines/PoisonUsage.txt /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/logs/UpdatedPoisonUsage.txt
```
It requires intimate knowledge of the pipeline to know that the new UpdatedPoisonUsage.txt file is captured as a build artifact.
One way to improve the UX would be to include the diff in the standard output.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.