CI Reliability 2023-09-12
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- node.js, shell
- Domain
- ci-cd, devops, testing-qa
Research direction
Start with the generated reliability report and the linked CI console output, then inspect the named tests under test/parallel and test/sequential. Compare failures across the listed operating systems and runners to determine whether they are reproducible. Done would require the reported recurring CI failures to be investigated and no longer block multiple pull requests.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/53804 to node-test-pull-request/53899 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-09-12 00:12 | 0 | 2 | 43 | 1 | 54 | 2.02% |
JSTest Failure
Example
not ok 672 parallel/test-runner-watch-mode
---
duration_ms: 3472.00200
severity: fail
exitcode: 1
stack: |-
TAP version 13
# Subtest: test runner watch mode
# Subtest: should run tests repeatedly
ok 1 - should run tests repeatedly
---
duration_ms: 1118.133392
...
| Reason | parallel/test-tls-socket-close |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/49279/, https://github.com/nodejs/node/pull/49567/, https://github.com/nodejs/node/pull/49532/, https://github.com/nodejs/node/pull/49144/, https://github.com/nodejs/node/pull/49543/) |
| Appeared | test-ibm-rhel8-s390x-2, test-ibm-rhel8-s390x-4, test-nearform-macos11.0-arm64-1, test-digitalocean-ubi81_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53815/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53887/ |
Example
not ok 2873 parallel/test-tls-socket-close
---
duration_ms: 424.85800
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at Immediate._onImmediate (/home/iojs/build/workspace/node-test-commit-linuxone/test/parallel/test-tls-socket-close.js:47:16)
at process.processImmediate (node:internal/timers:478:21) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v21.0.0-pre
...
| Reason | sequential/test-single-executable-application-snapshot-and-code-cache |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/49424/, https://github.com/nodejs/node/pull/49419/, https://github.com/nodejs/node/pull/49582/, https://github.com/nodejs/node/pull/49574/) |
| Appeared | test-azure_msft-win2016_vs2017-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53804/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53871/ |
Example
not ok 944 sequential/test-single-executable-application-snapshot-and-code-cache
---
duration_ms: 21083.99600
severity: fail
exitcode: 1
stack: |-
Aborted()
node:child_process:929
throw err;
^
Error: Command failed: C:\workspace\node-test-binary-windows-js-suites\node\Release\node.exe C:\workspace\node-test-binary-windows-js-suites\node\test\fixtures\postject-copy\node_modules\postject\dist\cli.js C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.943\sea.exe NODE_SEA_BLOB C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.943\sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
Aborted()
at checkExecSyncError (node:child_process:890:11)
at execFileSync (node:child_process:926:15)
at injectAndCodeSign (C:\workspace\node-test-binary-windows-js-suites\node\test\common\sea.js:48:3)
at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-singl...
| Reason | parallel/test-fs-watch-recursive-add-file-to-existing-subfolder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/49506/, https://github.com/nodejs/node/pull/49542/, https://github.com/nodejs/node/pull/49590/) |
| Appeared | test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53813/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53867/ |
Example
not ok 1384 parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
---
duration_ms: 120193.83600
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | sequential/test-cli-syntax-require |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/49419/, https://github.com/nodejs/node/pull/49568/, https://github.com/nodejs/node/pull/49569/) |
| Appeared | test-ibm-rhel8-x64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-ibm-aix72-ppc64_be-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53812/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53830/ |
Example
not ok 3916 sequential/test-cli-syntax-require
---
duration_ms: 183.53400
severity: fail
exitcode: 1
stack: |-
node:assert:1026
throw err;
^
AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^SyntaxError: \b/m. Input:
''
at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/sequential/test-cli-syntax-require.js:34:14)
at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/common/index.js:474:15)
at ChildProcess.emit (node:events:514:28)
at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: '',
expected: /^SyntaxError: \b/m,
operator: 'match'
}
Node.js v21.0.0-pre
...
| Reason | sequential/test-tls-psk-client |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/49574/, https://github.com/nodejs/node/pull/49587/, https://github.com/nodejs/node/pull/49611/) |
| Appeared | test-ibm-aix72-ppc64_be-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53839/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53895/ |
Example
not ok 3888 sequential/test-tls-psk-client
---
duration_ms: 5600.73800
severity: fail
exitcode: 1
stack: |-
Failed: Timed out
Failed: Error: connect ECONNREFUSED 127.0.0.1:12346
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
errno: -79,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 12346
}
...
| Reason | parallel/test-crypto-dh-generate-keys |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49542/, https://github.com/nodejs/node/pull/49237/) |
| Appeared | test-rackspace-win2012r2_vs2015-x64-2, test-azure_msft-win2016_vs2017-x64-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53825/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53889/ |
Example
not ok 916 parallel/test-crypto-dh-generate-keys
---
duration_ms: 120141.99800
severity: fail
stack: |-
timeout
...
| Reason | parallel/test-debugger-break |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49532/, https://github.com/nodejs/node/pull/49589/) |
| Appeared | test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2012r2_vs2019-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53851/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53875/ |
Example
not ok 165 parallel/test-debugger-break
---
duration_ms: 816.03400
severity: fail
exitcode: 1
stack: |-
node:internal/process/promises:288
triggerUncaughtException(err, true /* fromPromise */);
^
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert.ok(cli.output.includes(`#0 ${script}:6`))
at C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-debugger-break.js:63:10
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
Node.js v18.17.2-pre
...
| Reason | parallel/test-performance-function |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49543/, https://github.com/nodejs/node/pull/49587/) |
| Appeared | undefined |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53807/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53859/ |
Example
not ok 2248 parallel/test-performance-function
---
duration_ms: 265.32400
severity: fail
exitcode: 1
stack: |-
node:internal/event_target:1033
process.nextTick(() => { throw err; });
^
RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be a safe integer greater than 0. Received 0
at new NodeError (node:internal/errors:405:5)
at RecordableHistogram.record (node:internal/histogram:290:13)
at processComplete (node:internal/perf/timerify:40:15)
at timerified m (node:internal/perf/timerify:87:5)
at /home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/parallel/test-performance-function.js:98:5
Emitted 'error' event on Worker instance at:
at [kOnErrorMessage] (node:internal/worker:300:10)
at [kOnMessage] (node:internal/worker:311:37)
at MessagePort.<anonymous> (node:internal/worker:212:57)
at [nodejs.internal.kHybridDispatch] (node:interna...
| Reason | parallel/test-runner-output |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49279/, https://github.com/nodejs/node/pull/49563/) |
| Appeared | undefined, test-digitalocean-freebsd12-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53815/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53845/ |
Example
not ok 2641 parallel/test-runner-output
---
duration_ms: 3819.12300
severity: fail
exitcode: 1
stack: |-
TAP version 13
# Subtest: test runner output
# Subtest: test-runner/output/abort.js
ok 1 - test-runner/output/abort.js
---
duration_ms: 1459.806962
...
| Reason | parallel/test-tls-ticket-cluster |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49558/, https://github.com/nodejs/node/pull/49581/) |
| Appeared | test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53832/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53850/ |
Example
not ok 777 parallel/test-tls-ticket-cluster
---
duration_ms: 775.00000
severity: fail
exitcode: 1
stack: |-
[primary] got "listening"
[primary] connecting 64762 session? false
[primary] got "listening"
[primary] got "listening"
[primary] got "listening"
[worker] connection reused? false
[primary] got "not-reused"
[primary] connecting 64762 session? true
[primary] connecting 64762 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 64762 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 64762 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 64762 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 64762 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 64762 session? true
[worker] connection reused? true...
| Reason | sequential/test-http-regr-gh-2928 |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49570/, https://github.com/nodejs/node/pull/49543/) |
| Appeared | test-rackspace-win2012r2_vs2015-x64-1, test-rackspace-win2012r2_vs2015-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53834/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53887/ |
Example
not ok 934 sequential/test-http-regr-gh-2928
---
duration_ms: 639.98900
severity: fail
exitcode: 3221226505
stack: |-
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
...
| Reason | sequential/test-worker-fshandles-error-on-termination |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49534/, https://github.com/nodejs/node/pull/49531/) |
| Appeared | test-azure_msft-win2022_vs2019-x64-4, test-azure_msft-win2016_vs2017-x64-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53817/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53898/ |
Example
not ok 949 sequential/test-worker-fshandles-error-on-termination
---
duration_ms: 2245.36000
severity: fail
exitcode: 3221226356
stack: |-
...
| Reason | sequential/test-worker-fshandles-open-close-on-termination |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49546/, https://github.com/nodejs/node/pull/49542/) |
| Appeared | test-rackspace-win2019_vs2019-x64-4, test-azure_msft-win2016_vs2017-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53811/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53825/ |
Example
not ok 950 sequential/test-worker-fshandles-open-close-on-termination
---
duration_ms: 1274.03000
severity: fail
exitcode: 3221226356
stack: |-
...
| Reason | wpt/test-streams |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49546/, https://github.com/nodejs/node/pull/49542/) |
| Appeared | test-rackspace-win2022_vs2022-x64-2, test-rackspace-win2019_vs2019-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53809/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53823/ |
Example
not ok 954 wpt/test-streams
---
duration_ms: 3186.04900
severity: fail
exitcode: 3221226505
stack: |-
[SKIPPED] queuing-strategies-size-function-per-global.window.js: Browser-specific test
[SKIPPED] readable-streams\cross-realm-crash.window.js: Browser-specific test
[SKIPPED] readable-streams\owning-type-video-frame.any.js: Browser-specific test
[SKIPPED] readable-streams\read-task-handling.window.js: Browser-specific test
[SKIPPED] transferable\deserialize-error.window.js: Browser-specific test
[SKIPPED] transferable\transfer-with-messageport.window.js: Browser-specific test
[SKIPPED] transform-streams\invalid-realm.tentative.window.js: Browser-specific test
[PASS] a signal argument 'null' should cause pipeTo() to reject
[PASS] a signal argument 'AbortSignal' should cause pipeTo() to reject
[PASS] a signal argument 'true' should cause pipeTo() to reject
[PASS] a signal argument '-1' should cause pipeTo() to reject
[PASS] a signal argument '[object Abo...
| Reason | wpt/test-url |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49559/, https://github.com/nodejs/node/pull/49590/) |
| Appeared | test-rackspace-win2022_vs2022-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53819/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53886/ |
Example
not ok 954 wpt/test-url
---
duration_ms: 1920.55700
severity: crashed
exitcode: 3221225477
stack: |-
[SKIPPED] javascript-urls.window.js: requires document.body reference
[SKIPPED] percent-encoding.window.js: TODO: port from .window.js
[SKIPPED] url-setters-a-area.window.js?include=file: already tested in url-setters.any.js
[SKIPPED] url-setters-a-area.window.js?include=javascript: already tested in url-setters.any.js
[SKIPPED] url-setters-a-area.window.js?include=mailto: already tested in url-setters.any.js
[SKIPPED] url-setters-a-area.window.js?exclude=(file|javascript|mailto): already tested in url-setters.any.js
[EXPECTED_FAILURE][FAIL] searchParams on location object
assert_false: location object should not have a searchParams attribute expected false got true
at Test.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\fixtures\wpt\url\historical.any.js:3:5)
at Test.step (C:\workspace\node-test-binary-windows-js-suites\node\test...
Jenkins Failure
CCTest Failure
| Reason | C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected) |
|---|---|
| Type | CC_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49558/, https://github.com/nodejs/node/pull/49561/) |
| Appeared | test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53818/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53894/ |
Example
C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected)
Actual: true
Expected: false
[ FAILED ] InspectorSocketTest.HostIPv6NonRoutable (5001 ms)
Git Failure
Build Failure
| Reason | Failed to trigger sub builds |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49255/, https://github.com/nodejs/node/pull/48638/) |
| Appeared | undefined |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53857/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53897/ |
Example
Failed to trigger sub builds
| Reason | error: The last gc run reported the following. Please correct the root cause |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/49568/, https://github.com/nodejs/node/pull/49025/) |
| Appeared | test-ibm-ubuntu1804-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/53873/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/53880/ |
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
Progress
-
parallel/test-runner-watch-mode(6) -
parallel/test-tls-socket-close(5) -
sequential/test-single-executable-application-snapshot-and-code-cache(4) -
parallel/test-fs-watch-recursive-add-file-to-existing-subfolder(3) -
sequential/test-cli-syntax-require(3) -
sequential/test-tls-psk-client(3) -
parallel/test-crypto-dh-generate-keys(2) -
parallel/test-debugger-break(2) -
parallel/test-performance-function(2) -
parallel/test-runner-output(2) -
parallel/test-tls-ticket-cluster(2) -
sequential/test-http-regr-gh-2928(2) -
sequential/test-worker-fshandles-error-on-termination(2) -
sequential/test-worker-fshandles-open-close-on-termination(2) -
wpt/test-streams(2) -
wpt/test-url(2) -
C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected)(2) -
Failed to trigger sub builds(2) -
error: The last gc run reported the following. Please correct the root cause(2)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
out-of-date
Difficulty 1/5 Under an hour Newbie friendliness 82/100
CachyOS/CachyOS-PKGBUILDS#1894 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
community-scripts/ProxmoxVE#17396 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mattpocock/skills#1099 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100