CI Reliability 2023-02-26

Open
#507 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
nodejs, shell

Research direction

Start with the CI logs linked in the report and the failing tests under test/parallel, test/pummel, and node-api/test_threadsafe_function/test. Compare the listed failures across their first and last CI runs to determine whether they are reproducible or environment-specific. Done means each actionable failure has a confirmed fix or an explicitly tracked cause.

Written by the indexing model from the issue text.

Description

Failures in node-test-pull-request/49910 to node-test-pull-request/50002 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-02-26 00:17 7 4 29 10 50 4.82%
JSTest Failure
Reason parallel/test-http-remove-connection-header-persists-connection
Type JS_TEST_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/46753/, https://github.com/nodejs/node/pull/46796/, https://github.com/nodejs/node/pull/46795/, https://github.com/nodejs/node/pull/46735/, https://github.com/nodejs/node/pull/46802/, https://github.com/nodejs/node/pull/46806/, https://github.com/nodejs/node/pull/46536/, https://github.com/nodejs/node/pull/46815/, https://github.com/nodejs/node/pull/46816/)
Appeared test-digitalocean-ubi81_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-digitalocean-freebsd12-x64-1, test-digitalocean-alpine312_container-x64-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/49920/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49980/
Example
not ok 1370 parallel/test-http-remove-connection-header-persists-connection
  ---
  duration_ms: 0.611
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at /home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
        at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
        at listOnTimeout (node:internal/timers:568:17)
        at process.processTimers (node:internal/timers:511:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason parallel/test-http-remove-header-stays-removed
Type JS_TEST_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/46801/, https://github.com/nodejs/node/pull/46735/, https://github.com/nodejs/node/pull/46633/, https://github.com/nodejs/node/pull/46795/, https://github.com/nodejs/node/pull/46794/, https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/46816/, https://github.com/nodejs/node/pull/46818/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-5, test-digitalocean-alpine311_container-x64-2, test-softlayer-alpine312_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-softlayer-alpine311_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-alpine311_container-x64-1, test-digitalocean-alpine312_container-x64-1, test-digitalocean-ubi81_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-7, test-softlayer-ubuntu1804_sharedlibs_container-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/49933/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49983/
Example
not ok 1371 parallel/test-http-remove-header-stays-removed
  ---
  duration_ms: 0.259
  severity: fail
  exitcode: 1
  stack: |-
    ok
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-header-stays-removed.js:62:14)
        at listOnTimeout (node:internal/timers:568:17)
        at process.processTimers (node:internal/timers:511:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason pummel/test-net-throttle
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/44004/, https://github.com/nodejs/node/pull/46801/, https://github.com/nodejs/node/pull/46802/, https://github.com/nodejs/node/pull/46806/)
Appeared test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49915/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49946/
Example
not ok 3539 pummel/test-net-throttle
  ---
  duration_ms: 0.604
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
        at process.emit (node:events:522:35) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason node-api/test_threadsafe_function/test
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46806/, https://github.com/nodejs/node/pull/46818/)
Appeared test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49946/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49983/
Example
not ok 117 node-api/test_threadsafe_function/test
  ---
  duration_ms: 360.36
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-http-request-join-authorization-headers
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46767/, https://github.com/nodejs/node/pull/46330/)
Appeared test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-ibm-aix72-ppc64_be-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49922/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49961/
Example
not ok 1422 parallel/test-http-request-join-authorization-headers
  ---
  duration_ms: 0.257
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    '1, 2' !== '1'
    
        at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-request-join-authorization-headers.js:39:12)
        at Server.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/common/index.js:446:15)
        at Server.emit (node:events:513:28)
        at parserOnIncoming (node:_http_server:1072:12)
        at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: '1, 2',
      expected: '1',
      operator: 'strictEqual'
    }
    
    Node.js v18.14.3-pre
  ...


Reason parallel/test-single-executable-application
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46801/, https://github.com/nodejs/node/pull/46815/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/49933/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49978/
Example
not ok 2930 parallel/test-single-executable-application
  ---
  duration_ms: 120.128
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-wasm-web-api
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46753/, https://github.com/nodejs/node/pull/46815/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49923/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49978/
Example
not ok 3897 parallel/test-wasm-web-api
  ---
  duration_ms: 960.31
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason pummel/test-policy-integrity-parent-no-package-json
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46753/, https://github.com/nodejs/node/pull/46794/)
Appeared test-macstadium-macos11.0-arm64-3, test-macstadium-macos11.0-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/49923/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49952/
Example
not ok 3546 pummel/test-policy-integrity-parent-no-package-json
  ---
  duration_ms: 0.927
  severity: crashed
  exitcode: -10
  stack: |-
  ...


CCTest Failure
Jenkins Failure
Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46794/, https://github.com/nodejs/node/pull/46804/, https://github.com/nodejs/node/pull/46588/)
Appeared test-orka-macos11-x64-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-2, test-ibm-aix72-ppc64_be-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49917/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49969/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 199.7.167.101/199.7.167.101:32231' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
	at jdk.proxy2/jdk.proxy2.$Proxy79.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)

Reason Build timed out (after 30 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46767/)
Appeared test-azure_msft-win2016_vs2017-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/49917/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49918/
Example
Build timed out (after 30 minutes). Marking the build as failed.

Build Failure
Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/46770/, https://github.com/nodejs/node/pull/46767/, https://github.com/nodejs/node/pull/46795/, https://github.com/nodejs/node/pull/46804/, https://github.com/nodejs/node/pull/46818/, https://github.com/nodejs/node/pull/46815/)
Appeared test-rackspace-win2012r2_vs2017-x64-1, test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/49915/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49995/
Example
error: remote GitHub already exists.

> git fetch --prune --no-tags GitHub 

> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp

Reason error: 'test did not finish before its parent and was cancelled'
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44004/, https://github.com/nodejs/node/pull/46330/)
Appeared test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8
First CI https://ci.nodejs.org/job/node-test-pull-request/49930/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49961/
Example
error: 'test did not finish before its parent and was cancelled'
          code: 'ERR_TEST_FAILURE'
          ...
        # Subtest: should watch changes to dependencies - cjs
        not ok 5 - should watch changes to dependencies - cjs
          ---

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46801/)
Appeared test-equinix-rhel8_container-arm64-1, test-orka-macos10.14-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49912/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49933/
Example
fatal: No rebase in progress?

undefined
Reason Unknown
Type undefined
Failed PR 5 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46501/, https://github.com/nodejs/node/pull/46365/, https://github.com/nodejs/node/pull/46330/, https://github.com/nodejs/node/pull/46767/)
Appeared test-nearform_arm-win10_vs2019-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49927/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50002/
Example
Unknown

Git Failure
Reason warning: failed to remove Release/node.exe: Invalid argument
Type GIT_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/46768/, https://github.com/nodejs/node/pull/46501/, https://github.com/nodejs/node/pull/46365/, https://github.com/nodejs/node/pull/46330/, https://github.com/nodejs/node/pull/46767/)
Appeared test-rackspace-win2012r2_vs2015-x64-1, test-azure_msft-win2016_vs2017-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/49927/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50002/
Example
warning: failed to remove Release/node.exe: Invalid argument

Progress
  • parallel/test-http-remove-connection-header-persists-connection (10)
  • parallel/test-http-remove-header-stays-removed (8)
  • pummel/test-net-throttle (5)
  • node-api/test_threadsafe_function/test (2)
  • parallel/test-http-request-join-authorization-headers (2)
  • parallel/test-single-executable-application (2)
  • parallel/test-wasm-web-api (2)
  • pummel/test-policy-integrity-parent-no-package-json (2)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (4)
  • Build timed out (after 30 minutes). Marking the build as failed. (2)
  • error: remote GitHub already exists. (6)
  • error: 'test did not finish before its parent and was cancelled' (2)
  • fatal: No rebase in progress? (2)
  • Unknown (5)
  • warning: failed to remove Release/node.exe: Invalid argument (5)
Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from nodejs/reliability

All issues in nodejs/reliability

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.