CI Reliability 2023-02-26
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- nodejs, shell
- Domain
- ci-cd, devops, testing-qa
Research direction
Start with the CI logs linked in the report and the failing tests under test/parallel, test/pummel, and node-api/test_threadsafe_function/test. Compare the listed failures across their first and last CI runs to determine whether they are reproducible or environment-specific. Done means each actionable failure has a confirmed fix or an explicitly tracked cause.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/49910 to node-test-pull-request/50002 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-02-26 00:17 | 7 | 4 | 29 | 10 | 50 | 4.82% |
JSTest Failure
Example
not ok 1370 parallel/test-http-remove-connection-header-persists-connection
---
duration_ms: 0.611
severity: fail
exitcode: 1
stack: |-
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at /home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
at listOnTimeout (node:internal/timers:568:17)
at process.processTimers (node:internal/timers:511:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v20.0.0-pre
...
Example
not ok 1371 parallel/test-http-remove-header-stays-removed
---
duration_ms: 0.259
severity: fail
exitcode: 1
stack: |-
ok
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-header-stays-removed.js:62:14)
at listOnTimeout (node:internal/timers:568:17)
at process.processTimers (node:internal/timers:511:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v20.0.0-pre
...
| Reason | pummel/test-net-throttle |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/44004/, https://github.com/nodejs/node/pull/46801/, https://github.com/nodejs/node/pull/46802/, https://github.com/nodejs/node/pull/46806/) |
| Appeared | test-digitalocean-freebsd12-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49915/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49946/ |
Example
not ok 3539 pummel/test-net-throttle
---
duration_ms: 0.604
severity: fail
exitcode: 1
stack: |-
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
at process.emit (node:events:522:35) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v20.0.0-pre
...
| Reason | node-api/test_threadsafe_function/test |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46806/, https://github.com/nodejs/node/pull/46818/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49946/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49983/ |
Example
not ok 117 node-api/test_threadsafe_function/test
---
duration_ms: 360.36
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-http-request-join-authorization-headers |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46767/, https://github.com/nodejs/node/pull/46330/) |
| Appeared | test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-ibm-aix72-ppc64_be-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49922/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49961/ |
Example
not ok 1422 parallel/test-http-request-join-authorization-headers
---
duration_ms: 0.257
severity: fail
exitcode: 1
stack: |-
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
'1, 2' !== '1'
at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-request-join-authorization-headers.js:39:12)
at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/index.js:446:15)
at Server.emit (node:events:513:28)
at parserOnIncoming (node:_http_server:1072:12)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: '1, 2',
expected: '1',
operator: 'strictEqual'
}
Node.js v18.14.3-pre
...
| Reason | parallel/test-single-executable-application |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46801/, https://github.com/nodejs/node/pull/46815/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-5 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49933/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49978/ |
Example
not ok 2930 parallel/test-single-executable-application
---
duration_ms: 120.128
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-wasm-web-api |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46753/, https://github.com/nodejs/node/pull/46815/) |
| Appeared | test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49923/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49978/ |
Example
not ok 3897 parallel/test-wasm-web-api
---
duration_ms: 960.31
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | pummel/test-policy-integrity-parent-no-package-json |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46753/, https://github.com/nodejs/node/pull/46794/) |
| Appeared | test-macstadium-macos11.0-arm64-3, test-macstadium-macos11.0-arm64-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49923/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49952/ |
Example
not ok 3546 pummel/test-policy-integrity-parent-no-package-json
---
duration_ms: 0.927
severity: crashed
exitcode: -10
stack: |-
...
CCTest Failure
Jenkins Failure
| Reason | Backing channel 'JNLP4-connect connection from ... is disconnected. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46794/, https://github.com/nodejs/node/pull/46804/, https://github.com/nodejs/node/pull/46588/) |
| Appeared | test-orka-macos11-x64-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-2, test-ibm-aix72-ppc64_be-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49917/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49969/ |
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 199.7.167.101/199.7.167.101:32231' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at jdk.proxy2/jdk.proxy2.$Proxy79.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)
| Reason | Build timed out (after 30 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46767/) |
| Appeared | test-azure_msft-win2016_vs2017-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49917/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49918/ |
Example
Build timed out (after 30 minutes). Marking the build as failed.
Build Failure
| Reason | error: remote GitHub already exists. |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 6 (https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/46767/, https://github.com/nodejs/node/pull/46795/, https://github.com/nodejs/node/pull/46804/, https://github.com/nodejs/node/pull/46818/, https://github.com/nodejs/node/pull/46815/) |
| Appeared | test-rackspace-win2012r2_vs2017-x64-1, test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49915/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49995/ |
Example
error: remote GitHub already exists.
> git fetch --prune --no-tags GitHub
> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp
| Reason | error: 'test did not finish before its parent and was cancelled' |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/44004/, https://github.com/nodejs/node/pull/46330/) |
| Appeared | test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49930/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49961/ |
Example
error: 'test did not finish before its parent and was cancelled'
code: 'ERR_TEST_FAILURE'
...
# Subtest: should watch changes to dependencies - cjs
not ok 5 - should watch changes to dependencies - cjs
---
| Reason | fatal: No rebase in progress? |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46801/) |
| Appeared | test-equinix-rhel8_container-arm64-1, test-orka-macos10.14-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49912/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/49933/ |
Example
fatal: No rebase in progress?
undefined
Example
Unknown
Git Failure
| Reason | warning: failed to remove Release/node.exe: Invalid argument |
|---|---|
| Type | GIT_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46501/, https://github.com/nodejs/node/pull/46365/, https://github.com/nodejs/node/pull/46330/, https://github.com/nodejs/node/pull/46767/) |
| Appeared | test-rackspace-win2012r2_vs2015-x64-1, test-azure_msft-win2016_vs2017-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/49927/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/50002/ |
Example
warning: failed to remove Release/node.exe: Invalid argument
Progress
-
parallel/test-http-remove-connection-header-persists-connection(10) -
parallel/test-http-remove-header-stays-removed(8) -
pummel/test-net-throttle(5) -
node-api/test_threadsafe_function/test(2) -
parallel/test-http-request-join-authorization-headers(2) -
parallel/test-single-executable-application(2) -
parallel/test-wasm-web-api(2) -
pummel/test-policy-integrity-parent-no-package-json(2) -
Backing channel 'JNLP4-connect connection from ... is disconnected.(4) -
Build timed out (after 30 minutes). Marking the build as failed.(2) -
error: remote GitHub already exists.(6) -
error: 'test did not finish before its parent and was cancelled'(2) -
fatal: No rebase in progress?(2) -
Unknown(5) -
warning: failed to remove Release/node.exe: Invalid argument(5)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
community-scripts/ProxmoxVE#17396 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mattpocock/skills#1099 ·
-
Update Vish to 1.1.5 Openpackage-update
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
oSoWoSo/vOid_Community_repOsitory#144 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
help wanted new command
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
tldr-pages/tldr#24151 ·