CI Reliability 2022-07-19
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- node.js, shell
- Domain
- ci-cd, devops, testing-qa
Research direction
Start with the listed failing tests, including test/pummel/test-http-upload-timeout.js, test/sequential/test-gc-http-client.js, test/sequential/test-gc-http-client-onerror.js, test/parallel/test-worker-heap-snapshot.js, and test/pummel/test-fs-readfile-tostring-fail.js, using the linked CI examples. Determine the causes across the reported platforms and confirm that the affected CI failures no longer recur.
Written by the indexing model from the issue text.
Description
Failures in node-test-pull-request/45442 to node-test-pull-request/45540 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2022-07-19 00:18 | 0 | 4 | 23 | 1 | 72 | 4.04% |
JSTest Failure
Example
not ok 3328 pummel/test-http-upload-timeout
---
duration_ms: 5.730
severity: fail
exitcode: 1
stack: |-
/home/iojs/build/workspace/node-test-commit-arm/test/pummel/test-http-upload-timeout.js:34
throw new Error('Unexpected timeout');
^
Error: Unexpected timeout
at Socket.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm/test/pummel/test-http-upload-timeout.js:34:11)
at Socket.emit (node:events:525:35)
at Socket._onTimeout (node:net:522:8)
at listOnTimeout (node:internal/timers:564:17)
at process.processTimers (node:internal/timers:507:7)
Node.js v19.0.0-pre
...
Example
not ok 3425 sequential/test-gc-http-client
---
duration_ms: 1.717
severity: fail
exitcode: 1
stack: |-
done/collected/total: 160/0/640
done/collected/total: 320/160/640
done/collected/total: 480/256/640
done/collected/total: 639/320/640
done/collected/total: 640/480/640
done/collected/total: 640/639/640
done/collected/total: 640/640/640
node:_http_server:493
if (connections[i].socket._httpMessage && !connections[i].socket._httpMessage.finished) {
^
TypeError: Cannot read properties of null (reading '_httpMessage')
at Server.closeIdleConnections (node:_http_server:493:31)
at Server.close (node:_http_server:476:8)
at Immediate.status (/home/iojs/build/workspace/node-test-binary-armv7l/test/sequential/test-gc-http-client.js:60:14)
at process.processImmediate (node:internal/timers:471:21)
Node.js v19.0.0-pre
...
Example
not ok 3425 sequential/test-gc-http-client-onerror
---
duration_ms: 2.621
severity: fail
exitcode: 1
stack: |-
done/collected/total: 160/0/640
done/collected/total: 320/160/640
done/collected/total: 480/256/640
done/collected/total: 629/320/640
done/collected/total: 640/480/640
done/collected/total: 640/629/640
done/collected/total: 640/640/640
node:_http_server:493
connections[i].socket.destroy();
^
TypeError: Cannot read properties of null (reading 'destroy')
at Server.closeIdleConnections (node:_http_server:493:27)
at Server.close (node:_http_server:476:8)
at Immediate.status (/home/iojs/build/workspace/node-test-binary-armv7l/test/sequential/test-gc-http-client-onerror.js:65:14)
at process.processImmediate (node:internal/timers:471:21)
Node.js v19.0.0-pre
...
Example
not ok 3018 parallel/test-worker-heap-snapshot
---
duration_ms: 0.815
severity: crashed
exitcode: -11
stack: |-
...
| Reason | pummel/test-fs-readfile-tostring-fail |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 6 (https://github.com/nodejs/node/pull/43714/, https://github.com/nodejs/node/pull/43178/, https://github.com/nodejs/node/pull/43840/, https://github.com/nodejs/node/pull/43652/, https://github.com/nodejs/node/pull/43822/, https://github.com/nodejs/node/pull/43850/) |
| Appeared | test-digitalocean-freebsd12-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45444/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45461/ |
Example
not ok 3308 pummel/test-fs-readfile-tostring-fail
---
duration_ms: 28.608
severity: fail
exitcode: 7
stack: |-
/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-fs-readfile-tostring-fail.js:73
throw err;
^
AssertionError [ERR_ASSERTION]: 192675840 bytes written (expected 539555154 bytes).
at WriteStream.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-fs-readfile-tostring-fail.js:37:10)
at WriteStream.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/common/index.js:438:15)
at WriteStream.emit (node:events:513:28)
at finish (node:internal/streams/writable:748:10)
at finishMaybe (node:internal/streams/writable:733:9)
at afterWrite (node:internal/streams/writable:507:3)
at onwrite (node:internal/streams/writable:480:7)
at node:internal/fs/streams:443:5
at FSReqCall...
| Reason | parallel/test-http-client-response-timeout |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/43840/, https://github.com/nodejs/node/pull/43459/, https://github.com/nodejs/node/pull/43806/, https://github.com/nodejs/node/pull/43010/, https://github.com/nodejs/node/pull/43863/) |
| Appeared | test-ibm-rhel8-s390x-4, test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45453/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45485/ |
Example
not ok 1045 parallel/test-http-client-response-timeout
---
duration_ms: 0.70
severity: fail
exitcode: 1
stack: |-
Mismatched <anonymous> function calls. Expected exactly 1, actual 0.
at Proxy.mustCall (/home/iojs/build/workspace/node-test-commit-linuxone/test/common/index.js:396:10)
at ClientRequest.<anonymous> (/home/iojs/build/workspace/node-test-commit-linuxone/test/parallel/test-http-client-response-timeout.js:10:32)
at ClientRequest.<anonymous> (/home/iojs/build/workspace/node-test-commit-linuxone/test/common/index.js:438:15)
at Object.onceWrapper (node:events:628:26)
at ClientRequest.emit (node:events:513:28)
at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:674:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:117:17)
at Socket.socketOnData (node:_http_client:521:22)
at Socket.emit (node:events:513:28)
...
| Reason | parallel/test-vm-break-on-sigint |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/43840/, https://github.com/nodejs/node/pull/43891/, https://github.com/nodejs/node/pull/43890/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45453/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45527/ |
Example
not ok 3004 parallel/test-vm-break-on-sigint
---
duration_ms: 120.99
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | sequential/test-gc-http-client-timeout |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/43840/, https://github.com/nodejs/node/pull/43822/, https://github.com/nodejs/node/pull/43863/) |
| Appeared | test-nearform-macos11.0-arm64-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix-rhel8_container-arm64-1, test-equinix-ubuntu1804_container-arm64-2, test-equinix-ubuntu2004_container-arm64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45453/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45522/ |
Example
not ok 3612 sequential/test-gc-http-client-timeout
---
duration_ms: 0.448
severity: fail
exitcode: 1
stack: |-
done/collected/total: 48/0/503
done/collected/total: 97/48/503
done/collected/total: 166/97/503
node:events:491
throw er; // Unhandled 'error' event
^
Error: connect ECONNRESET ::1:64397
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1241:16)
Emitted 'error' event on ClientRequest instance at:
at Socket.socketErrorListener (node:_http_client:475:9)
at Socket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -54,
code: 'ECONNRESET',
syscall: 'connect',
address: '::1',
port: 64397
}
Node.js v19.0.0-pre
...
| Reason | parallel/test-fs-write-stream-file-handle |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43822/, https://github.com/nodejs/node/pull/43802/) |
| Appeared | test-rackspace-win2012r2_vs2017-x64-4, test-nearform-macos11.0-arm64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45518/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45533/ |
Example
not ok 290 parallel/test-fs-write-stream-file-handle
---
duration_ms: 0.869
severity: fail
exitcode: 1
stack: |-
Can't clean tmpdir: C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.282
Files blocking: [ 'write_stream_filehandle_test.txt' ]
node:assert:124
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected
+ ''
- 'hello world'
at WriteStream.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-fs-write-stream-file-handle.js:19:12)
at WriteStream.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:438:15)
at WriteStream.emit (node:events:513:28)
at emitCloseNT (node:internal/streams/destroy:132:10)
at process.processTicksAndRejections (node:internal/process/task_queues:81:21) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: '',
...
| Reason | parallel/test-gc-http-client-connaborted |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/42256/, https://github.com/nodejs/node/pull/43886/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-1, test-equinix-rhel8_container-arm64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45443/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45540/ |
Example
not ok 957 parallel/test-gc-http-client-connaborted
---
duration_ms: 2.58
severity: crashed
exitcode: -11
stack: |-
done/collected/total: 98/0/640
done/collected/total: 320/0/640
done/collected/total: 480/98/640
done/collected/total: 640/320/640
done/collected/total: 640/480/640
done/collected/total: 640/640/640
...
| Reason | parallel/test-heapsnapshot-near-heap-limit-worker |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43521/, https://github.com/nodejs/node/pull/43863/) |
| Appeared | test-rackspace-win2012r2_vs2019-x64-3, test-equinix-debian10_container-armv7l-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45452/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45485/ |
Example
not ok 324 parallel/test-heapsnapshot-near-heap-limit-worker
---
duration_ms: 3.808
severity: fail
exitcode: 1
stack: |-
Invoked NearHeapLimitCallback, processing=false, current_limit=52428800, initial_limit=52428800
max_young_gen_size=25165824, young_gen_size=0, old_gen_size=51120092, total_size=51120092
Estimated available memory=6695845888, estimated overhead=25165824
Start generating Heap.20220717.105101.2880.1.001.heapsnapshot...
Wrote snapshot to C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.287\Heap.20220717.105101.2880.1.001.heapsnapshot
<--- Last few GCs --->
[2880:069583C8] 2299 ms: Mark-sweep (reduce) 48.7 (52.8) -> 48.7 (52.8) MB, 5.7 / 0.0 ms (average mu = 0.320, current mu = -0.188) heap profiler; GC in old space requested
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 018AF82B v8::internal::Heap::PageFlagsAreC...
| Reason | parallel/test-performance-measure |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43763/, https://github.com/nodejs/node/pull/43863/) |
| Appeared | test-nearform-macos11.0-arm64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45459/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45522/ |
Example
not ok 1842 parallel/test-performance-measure
---
duration_ms: 1.259
severity: fail
exitcode: 1
stack: |-
node:assert:400
throw err;
^
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert.ok(duration > DELAY)
at /Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/parallel/test-performance-measure.js:12:12
at Array.forEach (<anonymous>)
at PerformanceObserver.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/parallel/test-performance-measure.js:11:22)
at PerformanceObserver.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/common/index.js:438:15)
at [kDispatch] (node:internal/perf/observe:314:19)
at Immediate._onImmediate (node:internal/perf/observe:118:25)
at process.processImmediate (node:internal/timers:471:21) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: fa...
Jenkins Failure
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from ip-83-147-191-75.broadband.digiweb.ie/83.147.191.75:60008' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy79.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
| Reason | Build timed out (after 15 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43714/, https://github.com/nodejs/node/pull/38905/) |
| Appeared | test-joyent-smartos20-x64-3 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45442/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45531/ |
Example
Build timed out (after 15 minutes). Marking the build as failed.
Build Failure
| Reason | ERROR: Build step failed with exception |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 7 (https://github.com/nodejs/node/pull/42256/, https://github.com/nodejs/node/pull/43822/, https://github.com/nodejs/node/pull/43763/, https://github.com/nodejs/node/pull/43010/, https://github.com/nodejs/node/pull/43747/, https://github.com/nodejs/node/pull/43870/, https://github.com/nodejs/node/pull/43886/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-2, test-equinix-debian10_container-armv7l-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45443/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45516/ |
Example
ERROR: Build step failed with exception
java.lang.NullPointerException: no workspace from node hudson.slaves.DumbSlave[test-equinix-ubuntu2004_container-armv7l-2] which is computer hudson.slaves.SlaveComputer@2ad6001 and has channel null
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:113)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
at org.jenkinsci.plugins.postbuildscript.processor.Processor.processBuildSteps(Processor.java:180)
at org.jenkinsci.plugins.postbuildscript.processor.Processor.processScripts(Processor.java:91)
| Reason | ERROR: Step ?Publish JUnit test result report? failed: no workspace for ... |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 5 (https://github.com/nodejs/node/pull/43763/, https://github.com/nodejs/node/pull/43010/, https://github.com/nodejs/node/pull/43884/, https://github.com/nodejs/node/pull/43886/, https://github.com/nodejs/node/pull/43802/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-2, test-equinix-ubuntu2004_container-armv7l-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45459/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45533/ |
Example
ERROR: Step ?Publish JUnit test result report? failed: no workspace for node-test-commit-arm/nodes=ubuntu2004-armv7l #42818
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE
| Reason | ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43850/, https://github.com/nodejs/node/pull/43889/) |
| Appeared | test-equinix-debian10_container-armv7l-2, test-equinix-ubuntu2004_container-armv7l-1, test-equinix-debian10_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45463/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45538/ |
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
[PostBuildScript] - [INFO] Executing post build scripts.
[node-test-binary-armv7l] $ /bin/bash -ex /tmp/jenkins10661375038590090938.sh
+ '[' -d .git ']'
+ git clean -fdx
| Reason | ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for ... |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/43010/, https://github.com/nodejs/node/pull/43892/) |
| Appeared | test-osuosl-rhel8-ppc64_le-2, test-digitalocean-freebsd12-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/45479/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/45536/ |
Example
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for node-test-commit-plinux/nodes=rhel8-ppc64le #43996
Sending e-mails to: michael_dawson@ca.ibm.com vieuxtech@gmail.com
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE
undefined
Example
Unknown
Progress
-
pummel/test-http-upload-timeout(20) -
sequential/test-gc-http-client(10) -
sequential/test-gc-http-client-onerror(7) -
parallel/test-worker-heap-snapshot(6) -
pummel/test-fs-readfile-tostring-fail(6) -
parallel/test-http-client-response-timeout(5) -
parallel/test-vm-break-on-sigint(3) -
sequential/test-gc-http-client-timeout(3) -
parallel/test-fs-write-stream-file-handle(2) -
parallel/test-gc-http-client-connaborted(2) -
parallel/test-heapsnapshot-near-heap-limit-worker(2) -
parallel/test-performance-measure(2) -
Backing channel 'JNLP4-connect connection from ... is disconnected.(11) -
Build timed out (after 15 minutes). Marking the build as failed.(2) -
ERROR: Build step failed with exception(7) -
ERROR: Step ?Publish JUnit test result report? failed: no workspace for ...(5) -
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?(2) -
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for ...(2) -
Unknown(25)
- Dominant language
- Shell
- Stars
- 57
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from nodejs/reliability
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1666 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
nodejs/reliability#1665 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
nodejs/reliability#1664 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1663 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
nodejs/reliability#1662 ·
All issues in nodejs/reliability
Similar issues
-
Update Vish to 1.1.5 Openpackage-update
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
oSoWoSo/vOid_Community_repOsitory#144 · 1 comment ·
-
help wanted new command
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
tldr-pages/tldr#24151 ·
-
PacReq: Rune IDE Openpackage request
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
pacstall/pacstall-programs#9402 ·
-
[BUG] ci: the first prefetch-images.sh call runs before shard.txt exists, so it is always a no-op Openbug github-actions
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
update-request
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
msys2/MINGW-packages#31768 ·