nodejs / nodejs/reliability

CI Reliability 2022-08-02

Open
#335 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/45684 to node-test-pull-request/45781 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2022-08-02 00:16 2 5 36 1 56 5.15%
Jenkins Failure
Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/44001/, https://github.com/nodejs/node/pull/44013/, https://github.com/nodejs/node/pull/43942/, https://github.com/nodejs/node/pull/44064/, https://github.com/nodejs/node/pull/44075/, https://github.com/nodejs/node/pull/44081/)
Appeared test-nearform-macos11.0-arm64-1, test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1, test-equinix-ubuntu2004_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/45693/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45772/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from ip-83-147-191-75.broadband.digiweb.ie/83.147.191.75:49506' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy79.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)

Git Failure
Build Failure
Reason ERROR: Build step failed with exception
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/43492/, https://github.com/nodejs/node/pull/43521/, https://github.com/nodejs/node/pull/43999/)
Appeared test-equinix-ubuntu2004_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/45688/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45704/
Example
ERROR: Build step failed with exception
java.lang.NullPointerException: no workspace from node hudson.slaves.DumbSlave[test-equinix-ubuntu2004_container-armv7l-2] which is computer hudson.slaves.SlaveComputer@2ad6001 and has channel null
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:113)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:91)
	at org.jenkinsci.plugins.postbuildscript.processor.Processor.processBuildSteps(Processor.java:180)
	at org.jenkinsci.plugins.postbuildscript.processor.Processor.processScripts(Processor.java:91)

Reason fatal error: ld terminated with signal 9 [Killed]
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44085/, https://github.com/nodejs/node/pull/44075/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-8
First CI https://ci.nodejs.org/job/node-test-pull-request/45774/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45776/
Example
fatal error: ld terminated with signal 9 [Killed]

undefined
Reason Unknown
Type undefined
Failed PR 19 (https://github.com/nodejs/node/pull/43629/, https://github.com/nodejs/node/pull/43977/, https://github.com/nodejs/node/pull/43492/, https://github.com/nodejs/node/pull/44001/, https://github.com/nodejs/node/pull/43984/, https://github.com/nodejs/node/pull/43999/, https://github.com/nodejs/node/pull/44013/, https://github.com/nodejs/node/pull/43521/, https://github.com/nodejs/node/pull/44018/, https://github.com/nodejs/node/pull/43784/, https://github.com/nodejs/node/pull/44048/, https://github.com/nodejs/node/pull/44032/, https://github.com/nodejs/node/pull/43942/, https://github.com/nodejs/node/pull/43904/, https://github.com/nodejs/node/pull/44056/, https://github.com/nodejs/node/pull/44077/, https://github.com/nodejs/node/pull/44060/, https://github.com/nodejs/node/pull/44080/, https://github.com/nodejs/node/pull/44081/)
Appeared test-nearform_arm-win10_vs2019-arm64-1, test-equinix-debian10_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/45685/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45781/
Example
Unknown

JSTest Failure
Reason parallel/test-worker-fshandles-error-on-termination
Type JS_TEST_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/43904/, https://github.com/nodejs/node/pull/43784/, https://github.com/nodejs/node/pull/44070/, https://github.com/nodejs/node/pull/44079/, https://github.com/nodejs/node/pull/44077/, https://github.com/nodejs/node/pull/44085/, https://github.com/nodejs/node/pull/44075/, https://github.com/nodejs/node/pull/44078/)
Appeared test-digitalocean-alpine312_container-x64-2, test-digitalocean-alpine311_container-x64-2, test-softlayer-alpine312_container-x64-1, test-digitalocean-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-7, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-softlayer-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-softlayer-ubuntu1804_sharedlibs_container-x64-3, test-softlayer-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-alpine311_container-x64-1, test-digitalocean-alpine312_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/45699/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45777/
Example
not ok 3055 parallel/test-worker-fshandles-error-on-termination
  ---
  duration_ms: 120.111
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-worker-fshandles-open-close-on-termination
Type JS_TEST_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/43904/, https://github.com/nodejs/node/pull/43784/, https://github.com/nodejs/node/pull/44070/, https://github.com/nodejs/node/pull/44079/, https://github.com/nodejs/node/pull/44077/, https://github.com/nodejs/node/pull/44085/, https://github.com/nodejs/node/pull/44075/, https://github.com/nodejs/node/pull/44078/)
Appeared test-digitalocean-alpine312_container-x64-2, test-digitalocean-alpine311_container-x64-2, test-softlayer-alpine312_container-x64-1, test-digitalocean-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-7, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-softlayer-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-softlayer-ubuntu1804_sharedlibs_container-x64-3, test-softlayer-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-alpine311_container-x64-1, test-digitalocean-alpine312_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/45699/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45777/
Example
not ok 3057 parallel/test-worker-fshandles-open-close-on-termination
  ---
  duration_ms: 120.107
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-worker-heap-snapshot
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/43629/, https://github.com/nodejs/node/pull/43784/, https://github.com/nodejs/node/pull/43984/, https://github.com/nodejs/node/pull/43521/, https://github.com/nodejs/node/pull/44032/, https://github.com/nodejs/node/pull/43904/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-rackspace-win2012r2_vs2019-x64-4, test-rackspace-win2012r2_vs2015-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-rackspace-win2012r2_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/45685/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45749/
Example
not ok 3036 parallel/test-worker-heap-snapshot
  ---
  duration_ms: 1.7
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-vm-break-on-sigint
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/43902/, https://github.com/nodejs/node/pull/44070/, https://github.com/nodejs/node/pull/44085/)
Appeared test-softlayer-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/45684/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45774/
Example
not ok 3049 parallel/test-vm-break-on-sigint
  ---
  duration_ms: 120.97
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-debugger-extract-function-name
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44077/, https://github.com/nodejs/node/pull/44080/)
Appeared test-azure_msft-win10_vs2019-x64-4, test-azure_msft-win10_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/45765/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45780/
Example
not ok 215 parallel/test-debugger-extract-function-name
  ---
  duration_ms: 6.571
  severity: fail
  exitcode: 1
  stack: |-
    C:\workspace\node-test-binary-windows-js-suites\node\test\common\debugger.js:84
              reject(new Error([
                     ^
    
    Error: Timeout (5000) while waiting for /break (?:on start )?in/i; found: connecting to 127.0.0.1:9229 ...
    < Debugger listening on ws://127.0.0.1:9229/ee606bb2-6224-44c1-8199-332d22636ee5
    < 
    
    < For help, see: https://nodejs.org/en/docs/inspector
    < 
    
     ok
    < Debugger attached.
    < 
    
    debug> 
        at Timeout.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\common\debugger.js:84:18)
        at listOnTimeout (node:internal/timers:564:17)
        at process.processTimers (node:internal/timers:507:7)
    
    Node.js v19.0.0-pre
  ...


Reason parallel/test-fs-stat-date
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44070/, https://github.com/nodejs/node/pull/44081/)
Appeared test-requireio_mininodes-debian10-armv7l_pi2-1, test-requireio_securogroup-debian10-arm64_pi3-1
First CI https://ci.nodejs.org/job/node-test-pull-request/45767/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45775/
Example
not ok 265 parallel/test-fs-stat-date
  ---
  duration_ms: 2.373
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/process/esm_loader:97
        internalBinding('errors').triggerUncaughtException(
                                  ^
    
    AssertionError [ERR_ASSERTION]: expected -40691 ± 1.0000000000000002, got 0
        at closeEnough (file:///home/iojs/build/workspace/node-test-binary-arm/test/parallel/test-fs-stat-date.mjs:26:10)
        at runTest (file:///home/iojs/build/workspace/node-test-binary-arm/test/parallel/test-fs-stat-date.mjs:40:3)
        at async file:///home/iojs/build/workspace/node-test-binary-arm/test/parallel/test-fs-stat-date.mjs:70:5 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
  ...


Reason sequential/test-worker-prof
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/43904/, https://github.com/nodejs/node/pull/44060/)
Appeared test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/45750/
Last CI https://ci.nodejs.org/job/node-test-pull-request/45768/
Example
not ok 879 sequential/test-worker-prof
  ---
  duration_ms: 2.680
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: child exited with non-zero status:                      {
      status: 3221225477,
      signal: null,
      output: [ null, '', '' ],
      pid: 10288,
      stdout: '',
      stderr: ''
    }
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-worker-prof.js:61:10)
        at Module._compile (node:internal/modules/cjs/loader:1120:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
        at Module.load (node:internal/modules/cjs/loader:998:32)
        at Module._load (node:internal/modules/cjs/loader:839:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
        at node:internal/main/run_main_module:17:47 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
 ...

Progress
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (6)
  • ERROR: Build step failed with exception (3)
  • fatal error: ld terminated with signal 9 [Killed] (2)
  • Unknown (19)
  • parallel/test-worker-fshandles-error-on-termination (8)
  • parallel/test-worker-fshandles-open-close-on-termination (8)
  • parallel/test-worker-heap-snapshot (6)
  • parallel/test-vm-break-on-sigint (3)
  • parallel/test-debugger-extract-function-name (2)
  • parallel/test-fs-stat-date (2)
  • sequential/test-worker-prof (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ncu-ci walk pr --stats=true --markdown report and inspect the linked Jenkins console logs for the listed JENKINS_FAILURE, BUILD_FAILURE, and JS_TEST_FAILURE entries. Compare the affected nodes and test names, then determine which recurring CI failures require action; done would mean the relevant reliability problem is addressed and the affected checks no longer fail repeatedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell
Domain
ci-cd, infrastructure, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.