CI Reliability 2023-03-13
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- git, node.js, shell
- Domain
- ci-cd, devops, testing-qa
Research direction
Start with the generated ncu-ci walk pr --stats=true --markdown report and its linked Jenkins output. Investigate the failures at test/cctest/test_inspector_socket.cc:323 and parallel/test-http-remove-connection-header-persists-connection, alongside the listed workspace, Git, and build errors. Done means the affected CI failures have identified root causes and no longer recur.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/50246 to node-test-pull-request/50340 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-03-13 00:15 | 0 | 2 | 32 | 5 | 61 | 2.11% |
CCTest Failure
| Reason | ../test/cctest/test_inspector_socket.cc:323: Failure |
|---|---|
| Type | CC_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46941/, https://github.com/nodejs/node/pull/46790/) |
| Appeared | test-softlayer-alpine312_container-x64-1, test-digitalocean-alpine311_container-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50250/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50261/ |
Example
../test/cctest/test_inspector_socket.cc:323: Failure
Value of: (!expectation.read_expected)
Actual: true
Expected: false
[ FAILED ] InspectorSocketTest.HostIPv6NonRoutable (5000 ms)
Jenkins Failure
| Reason | Cannot read file /home/iojs/.config/jgit/config |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 7 (https://github.com/nodejs/node/pull/47017/, https://github.com/nodejs/node/pull/47033/, https://github.com/nodejs/node/pull/46878/, https://github.com/nodejs/node/pull/46812/, https://github.com/nodejs/node/pull/47050/, https://github.com/nodejs/node/pull/46947/, https://github.com/nodejs/node/pull/47035/) |
| Appeared | test-osuosl-aix72-ppc64_be-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50310/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50336/ |
Example
java.io.IOException: Cannot read file /home/iojs/.config/jgit/config
org.eclipse.jgit.errors.ConfigInvalidException: Invalid line in config file
at org.eclipse.jgit.lib.Config.fromTextRecurse(Config.java:1206)
at org.eclipse.jgit.lib.Config.fromText(Config.java:1134)
at org.eclipse.jgit.storage.file.FileBasedConfig.lambda$0(FileBasedConfig.java:138)
at org.eclipse.jgit.util.FileUtils.readWithRetries(FileUtils.java:722)
| Reason | Build timed out (after 60 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/46615/, https://github.com/nodejs/node/pull/47017/, https://github.com/nodejs/node/pull/47028/, https://github.com/nodejs/node/pull/47039/) |
| Appeared | test-softlayer-ubi81_container-x64-1, test-rackspace-win2012r2_vs2019-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50289/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50340/ |
Example
Build timed out (after 60 minutes). Marking the build as failed.
| Reason | Backing channel 'JNLP4-connect connection from ... is disconnected. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/46809/, https://github.com/nodejs/node/pull/46815/, https://github.com/nodejs/node/pull/47050/) |
| Appeared | test-digitalocean-freebsd12-x64-1, test-equinix-ubuntu2004_container-armv7l-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50269/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50332/ |
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 45.55.90.237/45.55.90.237:46714' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at jdk.proxy2/jdk.proxy2.$Proxy209.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)
| Reason | Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46947/, https://github.com/nodejs/node/pull/46809/) |
| Appeared | test-orka-macos10.14-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50329/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50330/ |
Example
java.io.IOException: Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline
at hudson.matrix.MatrixRun$MatrixRunExecution.getParentWorkspaceLease(MatrixRun.java:173)
at hudson.matrix.MatrixRun$MatrixRunExecution.decideWorkspace(MatrixRun.java:187)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:509)
at hudson.model.Run.execute(Run.java:1897)
at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
Git Failure
Build Failure
| Reason | FATAL: java.nio.channels.ClosedChannelException |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 6 (https://github.com/nodejs/node/pull/47017/, https://github.com/nodejs/node/pull/47030/, https://github.com/nodejs/node/pull/46809/, https://github.com/nodejs/node/pull/46692/, https://github.com/nodejs/node/pull/46561/, https://github.com/nodejs/node/pull/41439/) |
| Appeared | test-equinix-ubuntu2204-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50292/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50297/ |
Example
FATAL: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 145.40.96.123/145.40.96.123:43896
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
at hudson.remoting.Request.call(Request.java:199)
at hudson.remoting.Channel.call(Channel.java:999)
| Reason | Failed to trigger fanned build |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/47030/, https://github.com/nodejs/node/pull/46809/, https://github.com/nodejs/node/pull/46692/, https://github.com/nodejs/node/pull/46561/, https://github.com/nodejs/node/pull/41439/) |
| Appeared | undefined |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50293/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50297/ |
Example
Failed to trigger fanned build
| Reason | error: remote GitHub already exists. |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/46815/, https://github.com/nodejs/node/pull/46947/) |
| Appeared | test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2017-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50248/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50300/ |
Example
error: remote GitHub already exists.
> git fetch --prune --no-tags GitHub
> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp
| Reason | error: The last gc run reported the following. Please correct the root cause |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47001/, https://github.com/nodejs/node/pull/46969/) |
| Appeared | test-ibm-ubuntu1804-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50263/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50266/ |
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
| Reason | fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46947/, https://github.com/nodejs/node/pull/46692/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50331/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50339/ |
Example
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.
fatal: failed to run repack
JSTest Failure
Example
not ok 1194 parallel/test-http-remove-connection-header-persists-connection
---
duration_ms: 0.238
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
at Timeout._onTimeout (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v20.0.0-pre
...
Example
not ok 3516 parallel/test-worker-nearheaplimit-deadlock
---
duration_ms: 120.33
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | sequential/test-child-process-pass-fd |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/46658/, https://github.com/nodejs/node/pull/46692/) |
| Appeared | test-macstadium-macos11.0-arm64-3, test-macstadium-macos11.0-arm64-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50248/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50301/ |
Example
not ok 3780 sequential/test-child-process-pass-fd
---
duration_ms: 0.254
severity: crashed
exitcode: -10
stack: |-
...
| Reason | sequential/test-http-regr-gh-2928 |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/46969/, https://github.com/nodejs/node/pull/46812/, https://github.com/nodejs/node/pull/47039/) |
| Appeared | test-digitalocean-freebsd12-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50267/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50338/ |
Example
not ok 3863 sequential/test-http-regr-gh-2928
---
duration_ms: 120.47
severity: fail
exitcode: -15
stack: |-
timeout
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
...
| Reason | parallel/test-fs-watch-recursive-add-file-with-url |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46812/, https://github.com/nodejs/node/pull/46947/) |
| Appeared | test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50299/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50300/ |
Example
not ok 1272 parallel/test-fs-watch-recursive-add-file-with-url
---
duration_ms: 120.121
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-fs-watch-recursive-add-folder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47012/, https://github.com/nodejs/node/pull/46947/) |
| Appeared | test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50278/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50300/ |
Example
not ok 1271 parallel/test-fs-watch-recursive-add-folder
---
duration_ms: 120.17
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-http-remove-header-stays-removed |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46812/, https://github.com/nodejs/node/pull/46947/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-alpine311_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50320/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50326/ |
Example
not ok 1372 parallel/test-http-remove-header-stays-removed
---
duration_ms: 0.232
severity: fail
exitcode: 1
stack: |-
ok
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-header-stays-removed.js:62:14)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v20.0.0-pre
...
| Reason | parallel/test-tls-ticket-cluster |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46692/, https://github.com/nodejs/node/pull/47033/) |
| Appeared | test-rackspace-win2012r2_vs2015-x64-2, test-ibm-rhel8-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50295/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50311/ |
Example
not ok 749 parallel/test-tls-ticket-cluster
---
duration_ms: 0.864
severity: fail
exitcode: 1
stack: |-
[primary] got "listening"
[primary] connecting 65207 session? false
[primary] got "listening"
[primary] got "listening"
[primary] got "listening"
[worker] connection reused? false
[primary] got "not-reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 65207 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connect...
| Reason | sequential/test-debugger-object-type-remote-object |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/46812/) |
| Appeared | test-macstadium-macos11.0-arm64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50248/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50320/ |
Example
not ok 3818 sequential/test-debugger-object-type-remote-object
---
duration_ms: 3.877
severity: fail
exitcode: 1
stack: |-
Mismatched noop function calls. Expected exactly 1, actual 0.
at Proxy.mustCall (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/common/index.js:407:10)
at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/sequential/test-debugger-object-type-remote-object.js:42:14)
at Module._compile (node:internal/modules/cjs/loader:1287:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1341:10)
at Module.load (node:internal/modules/cjs/loader:1145:32)
at Module._load (node:internal/modules/cjs/loader:984:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
at node:internal/main/run_main_module:23:47
...
| Reason | sequential/test-debugger-pid |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/46812/) |
| Appeared | test-macstadium-macos11.0-arm64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/50248/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50320/ |
Example
not ok 3819 sequential/test-debugger-pid
---
duration_ms: 5.631
severity: fail
exitcode: 1
stack: |-
node:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^
AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Timeout (5000) while waiting for /break/; found: Warning: script 'alive.js' was not loaded yet.
debug>
Break on start in test/fixtures/debugger/three-lines.js:1
> 1 let x = 1;
2 x = x + 1;
3 module.exports = x;
debug>
at runTest (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/sequential/test-debugger-pid.js:28:12)
at Timeout.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/common/debugger.js:90:18)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7) {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: Error: Ti...
Progress
-
../test/cctest/test_inspector_socket.cc:323: Failure(2) -
Cannot read file /home/iojs/.config/jgit/config(7) -
Build timed out (after 60 minutes). Marking the build as failed.(4) -
Backing channel 'JNLP4-connect connection from ... is disconnected.(3) -
Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline(2) -
FATAL: java.nio.channels.ClosedChannelException(6) -
Failed to trigger fanned build(5) -
error: remote GitHub already exists.(3) -
error: The last gc run reported the following. Please correct the root cause(2) -
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository(2) -
parallel/test-http-remove-connection-header-persists-connection(10) -
parallel/test-worker-nearheaplimit-deadlock(7) -
sequential/test-child-process-pass-fd(3) -
sequential/test-http-regr-gh-2928(3) -
parallel/test-fs-watch-recursive-add-file-with-url(2) -
parallel/test-fs-watch-recursive-add-folder(2) -
parallel/test-http-remove-header-stays-removed(2) -
parallel/test-tls-ticket-cluster(2) -
sequential/test-debugger-object-type-remote-object(2) -
sequential/test-debugger-pid(2)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
Update Vish to 1.1.5 Openpackage-update
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
oSoWoSo/vOid_Community_repOsitory#144 · 1 comment ·
-
help wanted new command
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
tldr-pages/tldr#24151 ·
-
PacReq: Rune IDE Openpackage request
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pacstall/pacstall-programs#9402 ·
-
[BUG] ci: the first prefetch-images.sh call runs before shard.txt exists, so it is always a no-op Openbug github-actions
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
update-request
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
msys2/MINGW-packages#31768 ·