CI Reliability 2023-05-22
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 20/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- node.js, shell
- Domain
- ci-cd, devops, testing-qa
Research direction
Start with the generated CI report and its linked Jenkins logs, then inspect the named tests: test/parallel/test-runner-output.js, test/parallel/test-http-remove-connection-header-persists-connection.js, test/parallel/test-runner-watch-mode.js, test/pummel/test-net-throttle.js, and test/sequential/test-worker-prof.js. Reproduce the failures on the listed platforms and determine which, if any, require a code or CI change; done is not defined by the issue.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/51804 to node-test-pull-request/51900 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-05-22 00:14 | 1 | 4 | 36 | 4 | 55 | 4.21% |
CCTest Failure
Jenkins Failure
| Reason | Build timed out (after 60 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/48034/, https://github.com/nodejs/node/pull/48062/) |
| Appeared | test-digitalocean-ubi81_container-x64-1, test-softlayer-ubi81_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51852/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51867/ |
Example
Build timed out (after 60 minutes). Marking the build as failed.
Git Failure
Build Failure
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Sending e-mails to: michael_dawson@ca.ibm.com vieuxtech@gmail.com ashley.cripps@ibm.com
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE
JSTest Failure
| Reason | parallel/test-runner-output |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/48009/, https://github.com/nodejs/node/pull/48033/, https://github.com/nodejs/node/pull/48034/, https://github.com/nodejs/node/pull/47866/, https://github.com/nodejs/node/pull/48038/) |
| Appeared | test-digitalocean-freebsd12-x64-1, test-digitalocean-freebsd12-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51847/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51898/ |
Example
not ok 2427 parallel/test-runner-output
---
duration_ms: 13294.75000
severity: fail
exitcode: 1
stack: |-
TAP version 13
# Subtest: test runner output
# Subtest: test-runner/output/abort.js
ok 1 - test-runner/output/abort.js
---
duration_ms: 12706.747848
...
| Reason | parallel/test-http-remove-connection-header-persists-connection |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/47866/, https://github.com/nodejs/node/pull/47824/, https://github.com/nodejs/node/pull/47986/, https://github.com/nodejs/node/pull/48092/) |
| Appeared | test-softlayer-ubuntu1804_sharedlibs_container-x64-3, test-ibm-rhel8-s390x-3, test-orka-macos11-x64-2, test-orka-macos11-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51874/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51891/ |
Example
not ok 1482 parallel/test-http-remove-connection-header-persists-connection
---
duration_ms: 296.79700
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at /home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v21.0.0-pre
...
| Reason | parallel/test-runner-watch-mode |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/47867/, https://github.com/nodejs/node/pull/48036/, https://github.com/nodejs/node/pull/48078/, https://github.com/nodejs/node/pull/48091/) |
| Appeared | test-rackspace-win2012r2_vs2015-x64-2, test-rackspace-win2012r2_vs2017-x64-4, test-equinix-ubuntu2004_container-arm64-2, test-digitalocean-alpine311_container-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51805/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51895/ |
Example
not ok 887 parallel/test-runner-watch-mode
---
duration_ms: 120130.98700
severity: fail
exitcode: 1
stack: |-
timeout
TAP version 13
# Subtest: test runner watch mode
# Subtest: should run tests repeatedly
ok 1 - should run tests repeatedly
---
duration_ms: 482.893568
...
| Reason | pummel/test-net-throttle |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/48020/, https://github.com/nodejs/node/pull/47821/, https://github.com/nodejs/node/pull/48092/, https://github.com/nodejs/node/pull/48098/) |
| Appeared | test-digitalocean-freebsd12-x64-1, test-digitalocean-freebsd12-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51818/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51896/ |
Example
not ok 3587 pummel/test-net-throttle
---
duration_ms: 442.04200
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
false !== true
at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
at process.emit (node:events:523:35) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: 'strictEqual'
}
Node.js v21.0.0-pre
...
| Reason | sequential/test-worker-prof |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 4 (https://github.com/nodejs/node/pull/47919/, https://github.com/nodejs/node/pull/47576/, https://github.com/nodejs/node/pull/48078/, https://github.com/nodejs/node/pull/48038/) |
| Appeared | test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-4, test-azure_msft-win10_vs2019-x64-4, test-nearform-macos11.0-arm64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51837/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51899/ |
Example
not ok 927 sequential/test-worker-prof
---
duration_ms: 1564.73100
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: child exited with non-zero status: {
status: 3221226505,
signal: null,
output: [ null, '', '' ],
pid: 1068,
stdout: '',
stderr: ''
}
at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-worker-prof.js:61:10)
at Module._compile (node:internal/modules/cjs/loader:1255:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1309:10)
at Module.load (node:internal/modules/cjs/loader:1113:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
at node:internal/main/run_main_module:23:47 {
generatedMessage: false,
code: 'ERR_ASSERTIO...
Example
not ok 86 js-native-api/test_cannot_run_js/test
---
duration_ms: 514.38900
severity: crashed
exitcode: -6
stack: |-
...
| Reason | parallel/test-single-executable-application |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/47433/, https://github.com/nodejs/node/pull/47447/, https://github.com/nodejs/node/pull/47455/) |
| Appeared | test-orka-macos11-x64-2, test-orka-macos11-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-7, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51839/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51842/ |
Example
not ok 2755 parallel/test-single-executable-application
---
duration_ms: 120.88
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | sequential/test-inspector-debug-end |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/47821/, https://github.com/nodejs/node/pull/48034/, https://github.com/nodejs/node/pull/48033/) |
| Appeared | test-ibm-rhel7-s390x-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51850/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51853/ |
Example
not ok 3690 sequential/test-inspector-debug-end
---
duration_ms: 120.250
severity: fail
exitcode: -15
stack: |-
timeout
Test there's no crash stopping server that was not started
Test there's no crash stopping server without connecting
[err] Debugger listening on ws://127.0.0.1:36322/d4d6b8ed-50c7-4e25-a533-09f9b69698d4
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
Test there's no crash stopping server after connecting
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:38574/965d12a2-7dd3-4193-95e3-b43922a3dfdd
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[err] Debugger attached.
[err]
...
| Reason | parallel/test-fs-watch-recursive-symlink |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/48033/, https://github.com/nodejs/node/pull/48072/) |
| Appeared | test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51853/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51870/ |
Example
not ok 1328 parallel/test-fs-watch-recursive-symlink
---
duration_ms: 120027.42000
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-http-request-join-authorization-headers |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47433/, https://github.com/nodejs/node/pull/47447/) |
| Appeared | test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-7, test-softlayer-ubuntu1804_sharedlibs_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51839/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51841/ |
Example
not ok 1433 parallel/test-http-request-join-authorization-headers
---
duration_ms: 0.303
severity: fail
exitcode: 1
stack: |-
node:assert:125
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
'1, 2' !== '1'
at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-request-join-authorization-headers.js:39:12)
at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/index.js:447:15)
at Server.emit (node:events:513:28)
at parserOnIncoming (node:_http_server:1091:12)
at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: '1, 2',
expected: '1',
operator: 'strictEqual'
}
Node.js v18.16.1-pre
...
| Reason | parallel/test-inspector-console |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47821/, https://github.com/nodejs/node/pull/47866/) |
| Appeared | test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51850/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51886/ |
Example
not ok 1781 parallel/test-inspector-console
---
duration_ms: 15591.77600
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:44615/736cb216-9fc5-4a21-b730-1e68e6fc004b
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[err] Debugger attached.
[err]
Timed out waiting for matching notification (Runtime.consoleAPICalled)
1
...
| Reason | parallel/test-repl-preview |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47336/, https://github.com/nodejs/node/pull/47821/) |
| Appeared | test-ibm-rhel7-s390x-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51820/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51822/ |
Example
not ok 2348 parallel/test-repl-preview
---
duration_ms: 2.10
severity: fail
exitcode: 1
stack: |-
Testing foo
Testing foo
Testing foo
Testing foo
Testing foo
Testing koo
Testing koo
Testing koo
Testing a
Testing a
Testing a
Testing { b: 1 }['b'] === 1
Testing { b: 1 }['b'] === 1
Testing { b: 1 }['b'] === 1
Testing { b: 1 }['b'] === 1;
Testing { b: 1 }['b'] === 1;
Testing { b: 1 }['b'] === 1;
Testing koo
Testing koo
Testing { a: true }
Testing { a: true }
Testing { a: true }
Testing { a: true };
Testing { a: true };
Testing { a: true };
Testing { a: true};
Testing { a: true};
Testing { a: true};
Testing 1n + 2n
Testing 1n + 2n
Testing 1n + 2n
Testing {};1
Testing {};1
Testing {};1
Testing a
Testing a
Testing { b: 1 }['b'] === 1
Testing { b: 1 }['b'] === 1
Testing { b: 1 }['b'] === 1;
Testing { b: 1 }['b'] === 1;
Testing { a: ...
| Reason | parallel/test-tls-ticket-cluster |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/47821/, https://github.com/nodejs/node/pull/47866/) |
| Appeared | test-azure_msft-win10_vs2019-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/51851/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/51883/ |
Example
not ok 754 parallel/test-tls-ticket-cluster
---
duration_ms: 525.00000
severity: fail
exitcode: 1
stack: |-
[primary] got "listening"
[primary] connecting 60990 session? false
[primary] got "listening"
[primary] got "listening"
[primary] got "listening"
[primary] connecting 60990 session? false
[worker] connection reused? false
[primary] got "not-reused"
[primary] connecting 60990 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 60990 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 60990 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 60990 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 60990 session? true
[worker] connection reused? true
[primary] got "reused"
[primary] connecting 60990 session? true
[worker] connection reused? tru...
Example
not ok 3943 wpt/test-webcrypto
---
duration_ms: 67135.04800
severity: crashed
exitcode: -6
stack: |-
[SKIPPED] algorithm-discards-context.https.window.js: Not relevant in Node.js context
[SKIPPED] historical.any.js: Not relevant in Node.js context
[PASS] setup - define tests
[PASS] X25519 key derivation checks for all-zero value result with a key of order 0
[PASS] X25519 key derivation checks for all-zero value result with a key of order 1
[PASS] X25519 key derivation checks for all-zero value result with a key of order 8
[PASS] X25519 key derivation checks for all-zero value result with a key of order p-1 (order 2)
[PASS] X25519 key derivation checks for all-zero value result with a key of order p (=0, order 4)
[PASS] X25519 key derivation checks for all-zero value result with a key of order p+1 (=1, order 1)
[PASS] X448 key derivation checks for all-zero value result with a key of order 0
[PASS] X448 key derivation checks for all-zero value result with a...
Progress
-
Build timed out (after 60 minutes). Marking the build as failed.(2) -
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?(6) -
parallel/test-runner-output(5) -
parallel/test-http-remove-connection-header-persists-connection(4) -
parallel/test-runner-watch-mode(4) -
pummel/test-net-throttle(4) -
sequential/test-worker-prof(4) -
js-native-api/test_cannot_run_js/test(3) -
parallel/test-single-executable-application(3) -
sequential/test-inspector-debug-end(3) -
parallel/test-fs-watch-recursive-symlink(2) -
parallel/test-http-request-join-authorization-headers(2) -
parallel/test-inspector-console(2) -
parallel/test-repl-preview(2) -
parallel/test-tls-ticket-cluster(2) -
wpt/test-webcrypto(2)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
out-of-date
Difficulty 1/5 Under an hour Newbie friendliness 82/100
CachyOS/CachyOS-PKGBUILDS#1894 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
community-scripts/ProxmoxVE#17396 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
mattpocock/skills#1099 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100