CI Reliability 2023-11-09
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- javascript, node.js
- Domain
- ci-cd, devops, testing-qa
Research direction
Start by reviewing the linked node-test-pull-request jobs and their console examples, then reproduce the named tests such as parallel/test-tls-ticket-cluster and sequential/test-single-executable-application-snapshot on the affected platforms. Trace each failure to a cause and verify that the relevant CI jobs no longer report the listed failures.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/55410 to node-test-pull-request/55507 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-11-09 00:13 | 0 | 4 | 39 | 3 | 54 | 4.12% |
JSTest Failure
| Reason | parallel/test-tls-ticket-cluster |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50237/, https://github.com/nodejs/node/pull/50506/, https://github.com/nodejs/node/pull/50574/) |
| Appeared | test-rackspace-win2016_vs2015-x64-1, test-rackspace-win2016_vs2015-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55422/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55462/ |
Example
not ok 750 parallel/test-tls-ticket-cluster
---
duration_ms: 1064.98800
severity: fail
exitcode: 3221226505
stack: |-
[primary] got "listening"
[primary] connecting 49769 session? false
[primary] got "listening"
[primary] got "listening"
[primary] got "listening"
[worker] connection reused? false
[primary] got "not-reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 49769 session? true
[worker] connection reused? true
[primary] got "reused"
[pr...
| Reason | parallel/test-worker-http2-stream-terminate |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50558/, https://github.com/nodejs/node/pull/50574/, https://github.com/nodejs/node/pull/50594/) |
| Appeared | test-azure_msft-win2016_vs2017-x64-3, test-rackspace-win2019_vs2019-x64-3, test-azure_msft-win2016_vs2017-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55451/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55506/ |
Example
not ok 862 parallel/test-worker-http2-stream-terminate
---
duration_ms: 1412.10400
severity: fail
exitcode: 3221226356
stack: |-
...
| Reason | sequential/test-single-executable-application-snapshot |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50558/, https://github.com/nodejs/node/pull/50465/, https://github.com/nodejs/node/pull/50594/) |
| Appeared | test-rackspace-win2016_vs2015-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55451/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55506/ |
Example
not ok 947 sequential/test-single-executable-application-snapshot
---
duration_ms: 26637.87100
severity: fail
exitcode: 1
stack: |-
[process 1640]: --- stderr ---
Aborted()
[process 1640]: --- stdout ---
[36mStart injection of NODE_SEA_BLOB in C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.946\sea.exe...[0m
[31mError: Aborted(). Build with -sASSERTIONS for more info.[0m
[process 1640]: status = 1, signal = null
C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86
throw new Error(`${failures.join('\n')}`);
^
Error: - process terminated with status 1, expected 0
at logAndThrow (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86:11)
at expectSyncExit (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:91:5)
at spawnSyncAndExitWithoutError (C:\workspace\node-test-binary-windows-js-suites\node\test\commo...
| Reason | sequential/test-single-executable-application-snapshot-and-code-cache |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50465/, https://github.com/nodejs/node/pull/50629/, https://github.com/nodejs/node/pull/50594/) |
| Appeared | test-rackspace-win2016_vs2015-x64-1, test-equinix-ubuntu2004_container-arm64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55501/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55506/ |
Example
not ok 946 sequential/test-single-executable-application-snapshot-and-code-cache
---
duration_ms: 25635.85600
severity: fail
exitcode: 1
stack: |-
[process 6488]: --- stderr ---
Aborted()
[process 6488]: --- stdout ---
[36mStart injection of NODE_SEA_BLOB in C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.945\sea.exe...[0m
[31mError: Aborted(). Build with -sASSERTIONS for more info.[0m
[process 6488]: status = 1, signal = null
C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86
throw new Error(`${failures.join('\n')}`);
^
Error: - process terminated with status 1, expected 0
at logAndThrow (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86:11)
at expectSyncExit (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:91:5)
at spawnSyncAndExitWithoutError (C:\workspace\node-test-binary-windows-js-suites\...
| Reason | parallel/test-blob-file-backed |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50327/, https://github.com/nodejs/node/pull/50572/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55435/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55473/ |
Example
not ok 190 parallel/test-blob-file-backed
---
duration_ms: 691.32800
severity: fail
exitcode: 1
stack: |-
node:internal/per_context/domexception:53
ErrorCaptureStackTrace(this);
^
DOMException [NotReadableError]: The blob could not be read
at new DOMException (node:internal/per_context/domexception:53:5)
at __node_internal_ (node:internal/util:698:10)
at BlobReader.<anonymous> (node:internal/blob:291:25)
Node.js v22.0.0-pre
...
| Reason | parallel/test-fs-watch-recursive-add-file-to-existing-subfolder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50555/, https://github.com/nodejs/node/pull/50476/) |
| Appeared | test-digitalocean-alpine315_container-x64-1, test-ibm-rhel8-s390x-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55463/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55474/ |
Example
not ok 1332 parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
---
duration_ms: 120140.89100
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-runner-output |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50237/, https://github.com/nodejs/node/pull/50628/) |
| Appeared | test-digitalocean-freebsd12-x64-2, test-osuosl-aix72-ppc64_be-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55422/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55503/ |
Example
not ok 2361 parallel/test-runner-output
---
duration_ms: 27636.71400
severity: fail
exitcode: 1
stack: |-
TAP version 13
# Subtest: test runner output
# Subtest: test-runner/output/abort.js
ok 1 - test-runner/output/abort.js
---
duration_ms: 24279.686443
...
| Reason | parallel/test-string-decoder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50506/, https://github.com/nodejs/node/pull/50558/) |
| Appeared | test-rackspace-win2016_vs2015-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55425/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55451/ |
Example
not ok 717 parallel/test-string-decoder
---
duration_ms: 315.99900
severity: fail
exitcode: 1
stack: |-
node:assert:635
throw err;
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected
+ Comparison {}
- Comparison {
- code: 'ERR_STRING_TOO_LONG'
- }
at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-string-decoder.js:205:10)
at Module._compile (node:internal/modules/cjs/loader:1241:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
at Module.load (node:internal/modules/cjs/loader:1091:32)
at Module._load (node:internal/modules/cjs/loader:938:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
at node:internal/main/run_main_module:23:47 {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: RangeError: Array buffer allocat...
| Reason | parallel/test-worker-memory |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50465/, https://github.com/nodejs/node/pull/50594/) |
| Appeared | test-rackspace-win2022_vs2022-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55501/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55506/ |
Example
not ok 864 parallel/test-worker-memory
---
duration_ms: 2052.03200
severity: crashed
exitcode: 3221225477
stack: |-
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=7.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=6.5 (numWorkers=8)
run() called with n=5.5 (numWorkers=8)
run() called with n=5.5 (numWorkers=8)
run() called with n=5.5 (numWorkers=8)
run() called with n=5.5 (numWorkers=8)
run() called with n=5.5 (numWorker...
Jenkins Failure
| Reason | Backing channel 'JNLP4-connect connection from ... is disconnected. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/50531/, https://github.com/nodejs/node/pull/50552/, https://github.com/nodejs/node/pull/50582/, https://github.com/nodejs/node/pull/50115/) |
| Appeared | test-digitalocean-fedora38-x64-2, test-rackspace-win2022_vs2022-x64-3, test-orka-macos11-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55430/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55475/ |
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 134.209.172.40/134.209.172.40:56160' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at jdk.proxy2/jdk.proxy2.$Proxy166.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)
| Reason | Build timed out (after 60 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/50572/, https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50594/, https://github.com/nodejs/node/pull/50600/) |
| Appeared | test-rackspace-win2022_vs2022-x64-5 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55473/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55507/ |
Example
Build timed out (after 60 minutes). Marking the build as failed.
| Reason | Build timed out (after 120 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50492/, https://github.com/nodejs/node/pull/50466/) |
| Appeared | test-digitalocean-fedora32-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55444/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55453/ |
Example
Build timed out (after 120 minutes). Marking the build as failed.
Git Failure
Build Failure
Example
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.
fatal: failed to run repack
| Reason | fatal error: string_view: No such file or directory |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 7 (https://github.com/nodejs/node/pull/50327/, https://github.com/nodejs/node/pull/50443/, https://github.com/nodejs/node/pull/50466/, https://github.com/nodejs/node/pull/50492/, https://github.com/nodejs/node/pull/50476/, https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50629/) |
| Appeared | test-rackspace-centos7-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55435/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55504/ |
Example
fatal error: string_view: No such file or directory
Example
fatal: No rebase in progress?
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
[PostBuildScript] - [INFO] Executing post build scripts.
[node-test-binary-armv7l] $ /bin/bash -ex /tmp/jenkins1473109786227085313.sh
+ '[' -d .git ']'
+ git clean -fdx
| Reason | error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj] |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50552/, https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50558/) |
| Appeared | test-rackspace-win2022_vs2022-x64-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55443/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55449/ |
Example
error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
property-cell-tq-csa.cc
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include\tuple(622): fatal error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
scope-info-tq-csa.cc
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include\tuple(625): fatal error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
| Reason | ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error? |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50629/) |
| Appeared | test-digitalocean-freebsd12-x64-1, test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55475/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55504/ |
Example
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE
| Reason | error: The last gc run reported the following. Please correct the root cause |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50629/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55475/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55504/ |
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
| Reason | error: remote GitHub already exists. |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50354/, https://github.com/nodejs/node/pull/50629/) |
| Appeared | test-rackspace-win2022_vs2022-x64-1, test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2022_vs2022-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55441/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55504/ |
Example
error: remote GitHub already exists.
> git fetch --prune --no-tags GitHub
> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp
Progress
-
parallel/test-tls-ticket-cluster(3) -
parallel/test-worker-http2-stream-terminate(3) -
sequential/test-single-executable-application-snapshot(3) -
sequential/test-single-executable-application-snapshot-and-code-cache(3) -
parallel/test-blob-file-backed(2) -
parallel/test-fs-watch-recursive-add-file-to-existing-subfolder(2) -
parallel/test-runner-output(2) -
parallel/test-string-decoder(2) -
parallel/test-worker-memory(2) -
Backing channel 'JNLP4-connect connection from ... is disconnected.(4) -
Build timed out (after 60 minutes). Marking the build as failed.(4) -
Build timed out (after 120 minutes). Marking the build as failed.(2) -
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository(17) -
fatal error: string_view: No such file or directory(7) -
fatal: No rebase in progress?(5) -
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?(4) -
error C1060: compiler is out of heap space [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj](3) -
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?(2) -
error: The last gc run reported the following. Please correct the root cause(2) -
error: remote GitHub already exists.(2)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
out-of-date
Difficulty 1/5 Under an hour Newbie friendliness 82/100
CachyOS/CachyOS-PKGBUILDS#1894 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
community-scripts/ProxmoxVE#17396 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mattpocock/skills#1099 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100