akkadotnet / akkadotnet/Hyperion
Build system: failing status on Mono gets overridden
- Dominant language
- C#
- Stars
- 281
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
Example: http://petabridge-ci.cloudapp.net/viewLog.html?buildId=23610&buildTypeId=Hyperion_HyperionBuildsMono&tab=buildLog#_state=260,273&focus=273&guest=1
Has two test failures:
```
[21:29:10] [docker]
[21:29:10] [docker] Test run for /checkout/Hyperion.Tests/bin/Release/netcoreapp1.0/Hyperion.Tests.dll(.NETCoreApp,Version=v1.0)
[21:29:11] [docker] Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
[21:29:11] [docker] Copyright (c) Microsoft Corporation. All rights reserved.
[21:29:11] [docker]
[21:29:11] [docker] Starting test execution, please wait...
[21:29:13] [docker] [xUnit.net 00:00:01.0619402] Discovering: Hyperion.Tests
[21:29:13] [docker] [xUnit.net 00:00:01.3994812] Discovered: Hyperion.Tests
[21:29:14] [docker] [xUnit.net 00:00:01.5054871] Starting: Hyperion.Tests
[21:29:14] [docker] [xUnit.net 00:00:02.0633171] Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: True, val2: False) [FAIL]
[21:29:14] [docker] [xUnit.net 00:00:02.0670384] Assert.Equal() Failure
[21:29:14] [docker] [xUnit.net 00:00:02.0671854] Expected: False
[21:29:14] [docker] [xUnit.net 00:00:02.0672427] Actual: True
[21:29:14] [docker] [xUnit.net 00:00:02.0690800] Stack Trace:
[21:29:14] [docker] [xUnit.net 00:00:02.0782522] /checkout/Hyperion.Tests/CustomObjectTests.cs(87,0): at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2)
[21:29:14] [docker] [xUnit.net 00:00:02.0818967] Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: False, val2: True) [FAIL]
[21:29:14] [docker] [xUnit.net 00:00:02.0820607] Assert.Equal() Failure
[21:29:14] [docker] [xUnit.net 00:00:02.0821049] Expected: True
[21:29:14] [docker] [xUnit.net 00:00:02.0825918] Actual: False
[21:29:14] [docker] [xUnit.net 00:00:02.0826474] Stack Trace:
[21:29:14] [docker] [xUnit.net 00:00:02.0830364] /checkout/Hyperion.Tests/CustomObjectTests.cs(87,0): at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2)
[21:29:14] [docker] Failed Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: True, val2: False)
[21:29:14] [docker] Error Message:
[21:29:14] [docker] Assert.Equal() Failure
[21:29:14] [docker] Expected: False
[21:29:14] [docker] Actual: True
[21:29:14] [docker] Stack Trace:
[21:29:14] [docker] at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2) in /checkout/Hyperion.Tests/CustomObjectTests.cs:line 87
[21:29:14] [docker] Failed Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(val1: False, val2: True)
[21:29:14] [docker] Error Message:
[21:29:14] [docker] Assert.Equal() Failure
[21:29:14] [docker] Expected: True
[21:29:14] [docker] Actual: False
[21:29:14] [docker] Stack Trace:
[21:29:14] [docker] at Hyperion.Tests.CustomObjectTests.CanSerializeDefaultCtorArguments(Boolean val1, Boolean val2) in /checkout/Hyperion.Tests/CustomObjectTests.cs:line 87
[21:29:15] [docker] [xUnit.net 00:00:03.0937879] Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray [SKIP]
[21:29:15] [docker] [xUnit.net 00:00:03.0939669] add support for multi dimentional arrays
[21:29:15] [docker] Skipped Hyperion.Tests.CollectionTests.CanSerializeMultiDimentionalArray
[21:29:15] [docker] [xUnit.net 00:00:03.4139414] Finished: Hyperion.Tests
[21:29:16] [docker]
[21:29:16] [docker] Total tests: 119. Passed: 116. Failed: 2. Skipped: 1.
[21:29:16] [docker] Test Run Failed.
[21:29:16] [docker] Test execution time: 5.1388 Seconds
```
But still has a green status. Looks like it's "forgetting" that it failed or that the failure status is overridden. Probably a result of running the build twice. cc @heynickc
Contributor guide
Research direction
Start with the build configuration that runs Hyperion.Tests and compare the repeated build results against the linked CI log. Inspect Hyperion.Tests/CustomObjectTests.cs at line 87 to reproduce the two failures; done means a build containing failed tests remains red rather than being reported green.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, docker
- Domain
- build-system, ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100