CI Reliability 2022-03-08

Open
#214 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
node.js, shell
Domain
ci-cd, testing-qa

Research direction

Start with the failing test names and linked CI examples, especially test/parallel/test-crypto-key-objects and test/parallel/test-module-readonly, then compare their recorded failures across the listed platforms. The report does not define a specific fix; done would require identifying the causes of the repeated failures and restoring reliable CI results.

Written by the indexing model from the issue text.

Description

Failures in node-test-pull-request/42802 to node-test-pull-request/42899 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2022-03-08 00:10 2 21 16 8 53 23.33%
JSTest Failure
Reason parallel/test-crypto-key-objects
Type JS_TEST_FAILURE
Failed PR 9 (https://github.com/nodejs/node/pull/42056/, https://github.com/nodejs/node/pull/42204/, https://github.com/nodejs/node/pull/42212/, https://github.com/nodejs/node/pull/42131/, https://github.com/nodejs/node/pull/42062/, https://github.com/nodejs/node/pull/42234/, https://github.com/nodejs/node/pull/42236/, https://github.com/nodejs/node/pull/41513/, https://github.com/nodejs/node/pull/42246/)
Appeared test-softlayer-debian10-x64-1, test-rackspace-fedora32-x64-1, test-equinix-centos7_container-arm64-2, test-equinix-ubuntu1804_container-arm64-2, test-softlayer-alpine311_container-x64-1, test-equinix-ubuntu2004_container-arm64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-orka-macos11-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/42828/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42899/
Example
not ok 923 parallel/test-crypto-key-objects
  ---
  duration_ms: 120.89
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-module-readonly
Type JS_TEST_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/42117/, https://github.com/nodejs/node/pull/41677/, https://github.com/nodejs/node/pull/42203/, https://github.com/nodejs/node/pull/42204/, https://github.com/nodejs/node/pull/41513/, https://github.com/nodejs/node/pull/42177/, https://github.com/nodejs/node/pull/42190/, https://github.com/nodejs/node/pull/42246/)
Appeared test-nearform_arm-win10_vs2019-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/42808/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42899/
Example
not ok 465 parallel/test-module-readonly
  ---
  duration_ms: 1.88
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:991
        throw newErr;
        ^
    
    AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Cannot find module '..\.tmp.456\readOnlyMod'
    Require stack:
    - c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-module-readonly.js
        at Object.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-module-readonly.js:48:8)
        at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
        at Function.Module._load (node:internal/modules/cjs/loader:778:27)
        at Module.require (node:internal/modules/cjs/loader:999:19)
        at require (node:internal/modules/cjs/helpers:102:18)
        at Object.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-module-readonly.js:36:3)
        at Module._compile (node:internal/modules/cjs/loader:1097:14)
        at ...

Reason node-api/test_threadsafe_function/test
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/42163/, https://github.com/nodejs/node/pull/42112/, https://github.com/nodejs/node/pull/42131/, https://github.com/nodejs/node/pull/42159/)
Appeared test-equinix-ubuntu2004_container-arm64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-centos7_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/42846/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42883/
Example
not ok 3370 node-api/test_threadsafe_function/test
  ---
  duration_ms: 240.17
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-child-process-fork-closed-channel-segfault
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/42141/, https://github.com/nodejs/node/pull/42147/, https://github.com/nodejs/node/pull/42155/)
Appeared test-requireio_securogroup-debian10-arm64_pi3-2, test-requireio_joeyvandijk-debian10-armv7l_pi2-2, test-requireio_jasnell-debian10-armv7l_pi2-1
First CI https://ci.nodejs.org/job/node-test-pull-request/42813/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42825/
Example
not ok 104 parallel/test-child-process-fork-closed-channel-segfault
  ---
  duration_ms: 3.72
  severity: fail
  exitcode: 1
  stack: |-
    /home/iojs/build/workspace/node-test-binary-arm/test/parallel/test-child-process-fork-closed-channel-segfault.js:83
                throw err;
                ^
    
    Error: write EPIPE
        at ChildProcess.target._send (node:internal/child_process:864:20)
        at ChildProcess.<anonymous> (node:internal/child_process:667:16)
        at ChildProcess.emit (node:events:539:35)
        at emit (node:internal/child_process:936:14)
        at processTicksAndRejections (node:internal/process/task_queues:83:21) {
      errno: -32,
      code: 'EPIPE',
      syscall: 'write'
    }
    
    Node.js v18.0.0-pre
  ...


Reason parallel/test-crypto-dh
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/42147/, https://github.com/nodejs/node/pull/42056/, https://github.com/nodejs/node/pull/42212/)
Appeared test-requireio_williamkapke-debian10-arm64_pi3-1, test-requireio_williamkapke-debian10-arm64_pi3-2, test-requireio_svincent-debian10-armv7l_pi2-2
First CI https://ci.nodejs.org/job/node-test-pull-request/42817/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42872/
Example
not ok 293 parallel/test-crypto-dh
  ---
  duration_ms: 240.145
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-http-header-overflow
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/41677/, https://github.com/nodejs/node/pull/42177/, https://github.com/nodejs/node/pull/42190/)
Appeared test-rackspace-win2012r2_vs2019-x64-3, test-rackspace-win2012r2_vs2013-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/42814/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42873/
Example
not ok 313 parallel/test-http-header-overflow
  ---
  duration_ms: 0.218
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:635
          throw err;
          ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
    + actual - expected ... Lines skipped
    
      Comparison {
    +   bytesParsed: 10642,
    -   bytesParsed: 16402,
        code: 'HPE_HEADER_OVERFLOW',
        message: 'Parse Error: Header overflow',
        name: 'Error',
    +   rawPacket: Buffer(10645) [Uint8Array] [
    ...


Reason parallel/test-crypto-classes
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42161/, https://github.com/nodejs/node/pull/42247/)
Appeared test-requireio_svincent-debian10-armv7l_pi2-2, test-requireio_joeyvandijk-debian10-armv7l_pi2-2
First CI https://ci.nodejs.org/job/node-test-pull-request/42829/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42898/
Example
not ok 215 parallel/test-crypto-classes
  ---
  duration_ms: 240.82
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-crypto-dh-constructor
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42203/, https://github.com/nodejs/node/pull/42196/)
Appeared test-requireio_mininodes-debian10-armv7l_pi2-1, test-requireio_svincent-debian10-armv7l_pi2-2
First CI https://ci.nodejs.org/job/node-test-pull-request/42849/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42851/
Example
not ok 208 parallel/test-crypto-dh-constructor
  ---
  duration_ms: 240.79
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-net-connect-memleak
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42203/, https://github.com/nodejs/node/pull/42246/)
Appeared test-rackspace-win2012r2_vs2017-x64-4, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/42849/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42899/
Example
not ok 465 parallel/test-net-connect-memleak
  ---
  duration_ms: 0.243
  severity: fail
  exitcode: 1
  stack: |-
    Mismatched noop function calls. Expected exactly 1, actual 0.
        at Proxy.mustCall (C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:369:10)
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-net-connect-memleak.js:33:40)
        at Module._compile (node:internal/modules/cjs/loader:1099:14)
        at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
        at Module.load (node:internal/modules/cjs/loader:975:32)
        at Function.Module._load (node:internal/modules/cjs/loader:822:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
        at node:internal/main/run_main_module:17:47
  ...


Reason sequential/test-debugger-restart-message
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42141/, https://github.com/nodejs/node/pull/42062/)
Appeared test-requireio_jasnell-debian10-armv7l_pi2-1, test-requireio_svincent-debian10-armv7l_pi2-3
First CI https://ci.nodejs.org/job/node-test-pull-request/42805/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42884/
Example
not ok 835 sequential/test-debugger-restart-message
  ---
  duration_ms: 28.241
  severity: fail
  exitcode: 1
  stack: |-
    /home/iojs/build/workspace/node-test-binary-arm/test/common/debugger.js:84
              reject(new Error([
                     ^
    
    Error: Timeout (10000) while waiting for /(?:assert|break|break on start|debugCommand|exception|other|promiseRejection) in/i; found: 
    < Debugger ending on ws://127.0.0.1:9229/6d95b2c1-2dcf-4eae-98da-66ce07333961
    < For help, see: https://nodejs.org/en/docs/inspector
    < 
    debug> 
    
    < Debugger listening on ws://127.0.0.1:9229/c95d5cb3-5ea1-4466-9acc-9415f9fdfded
    < For help, see: https://nodejs.org/en/docs/inspector
    < 
    debug> 
    
    connecting to 127.0.0.1:9229 ...
     ok
    
    debug> 
    
    < Debugger attached.
    < 
    debug> 
        at Timeout.<anonymous> (/home/iojs/build/workspace/node-test-binary-arm/test/common/debugger.js:84:18)
        at listOnTimeout (node:internal/timers:559:17)
        at proc...

Jenkins Failure
Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42135/, https://github.com/nodejs/node/pull/42176/)
Appeared test-equinix-ubuntu2004_container-armv7l-1, test-ibm-rhel7-s390x-2, test-ibm-rhel7-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/42819/
Last CI https://ci.nodejs.org/job/node-test-pull-request/42875/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 145.40.81.219/145.40.81.219:39920' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
	at com.sun.proxy.$Proxy79.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)

Git Failure
Build Failure
Progress
  • parallel/test-crypto-key-objects (9)
  • parallel/test-module-readonly (8)
  • node-api/test_threadsafe_function/test (4)
  • parallel/test-child-process-fork-closed-channel-segfault (3)
  • parallel/test-crypto-dh (3)
  • parallel/test-http-header-overflow (3)
  • parallel/test-crypto-classes (2)
  • parallel/test-crypto-dh-constructor (2)
  • parallel/test-net-connect-memleak (2)
  • sequential/test-debugger-restart-message (2)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (2)
Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from nodejs/reliability

All issues in nodejs/reliability

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.