akkadotnet / akkadotnet/Hyperion

Build system: failing status on Mono gets overridden

Open
#39 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
281
Forks
62
PR merge metrics
No merged PRs in 30d

Description

Example: http://petabridge-ci.cloudapp.net/viewLog.html?buildId=23610&buildTypeId=Hyperion_HyperionBuildsMono&tab=buildLog#_state=260,273&focus=273&guest=1

Has two test failures:

```
[21:29:10] [docker]
[21:29:10] [docker] Test run for /checkout/Hyperion.Tests/bin/Release/netcoreapp1.0/Hyperion.Tests.dll(.NETCoreApp,Version=v1.0)
[21:29:11] [docker] Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
[21:29:11] [docker] Copyright (c) Microsoft Corporation. All rights reserved.
[21:29:11] [docker]
[21:29:11] [docker] Starting test execution, please wait...
[21:29:13] [docker] [xUnit.net 00:00:01.0619402] Discovering: Hyperion.Tests
[21:29:13] [docker] [xUnit.net 00:00:01.3994812] Discovered: Hyperion.Tests
[21:29:14] [docker] [xUnit.net 00:00:01.5054871] Starting: Hyperion.Tests
[21:29:14] [docker] [xUnit.net 00:00:02.0633171] Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: True, val2: False) [FAIL]
[21:29:14] [docker] [xUnit.net 00:00:02.0670384] Assert.Equal() Failure
[21:29:14] [docker] [xUnit.net 00:00:02.0671854] Expected: False
[21:29:14] [docker] [xUnit.net 00:00:02.0672427] Actual: True
[21:29:14] [docker] [xUnit.net 00:00:02.0690800] Stack Trace:
[21:29:14] [docker] [xUnit.net 00:00:02.0782522] /checkout/Hyperion.Tests/CustomObjectTests.cs(87,0): at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2)
[21:29:14] [docker] [xUnit.net 00:00:02.0818967] Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: False, val2: True) [FAIL]
[21:29:14] [docker] [xUnit.net 00:00:02.0820607] Assert.Equal() Failure
[21:29:14] [docker] [xUnit.net 00:00:02.0821049] Expected: True
[21:29:14] [docker] [xUnit.net 00:00:02.0825918] Actual: False
[21:29:14] [docker] [xUnit.net 00:00:02.0826474] Stack Trace:
[21:29:14] [docker] [xUnit.net 00:00:02.0830364] /checkout/Hyperion.Tests/CustomObjectTests.cs(87,0): at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2)
[21:29:14] [docker] Failed Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: True, val2: False)
[21:29:14] [docker] Error Message:
[21:29:14] [docker] Assert.Equal() Failure
[21:29:14] [docker] Expected: False
[21:29:14] [docker] Actual: True
[21:29:14] [docker] Stack Trace:
[21:29:14] [docker] at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2) in /checkout/Hyperion.Tests/CustomObjectTests.cs:line 87
[21:29:14] [docker] Failed Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: False, val2: True)
[21:29:14] [docker] Error Message:
[21:29:14] [docker] Assert.Equal() Failure
[21:29:14] [docker] Expected: True
[21:29:14] [docker] Actual: False
[21:29:14] [docker] Stack Trace:
[21:29:14] [docker] at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2) in /checkout/Hyperion.Tests/CustomObjectTests.cs:line 87
[21:29:15] [docker] [xUnit.net 00:00:03.0937879] Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray [SKIP]
[21:29:15] [docker] [xUnit.net 00:00:03.0939669] add support for multi dimentional arrays
[21:29:15] [docker] Skipped Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray
[21:29:15] [docker] [xUnit.net 00:00:03.4139414] Finished: Hyperion.Tests
[21:29:16] [docker]
[21:29:16] [docker] Total tests: 119. Passed: 116. Failed: 2. Skipped: 1.
[21:29:16] [docker] Test Run Failed.
[21:29:16] [docker] Test execution time: 5.1388 Seconds
```

But still has a green status. Looks like it's "forgetting" that it failed or that the failure status is overridden. Probably a result of running the build twice. cc @heynickc

Contributor guide

Open the contributing guide

Research direction

Start with the build configuration that runs Hyperion.Tests and compare the repeated build results against the linked CI log. Inspect Hyperion.Tests/CustomObjectTests.cs at line 87 to reproduce the two failures; done means a build containing failed tests remains red rather than being reported green.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, docker
Domain
build-system, ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.