nodejs / nodejs/reliability

CI Reliability 2023-03-19

Open
#528 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/50353 to node-test-pull-request/50449 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-03-19 00:16 2 1 30 5 62 1.08%
CCTest Failure
Jenkins Failure
Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/46692/, https://github.com/nodejs/node/pull/47079/, https://github.com/nodejs/node/pull/47011/, https://github.com/nodejs/node/pull/46941/)
Appeared test-softlayer-ubi81_container-x64-1, test-digitalocean-ubi81_container-x64-1, test-digitalocean-ubi81_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/50365/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50447/
Example
Build timed out (after 60 minutes). Marking the build as failed.

Reason Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline
Type JENKINS_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/46688/, https://github.com/nodejs/node/pull/47039/, https://github.com/nodejs/node/pull/47132/, https://github.com/nodejs/node/pull/47139/)
Appeared test-orka-macos10.14-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50359/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50421/
Example
java.io.IOException: Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline
	at hudson.matrix.MatrixRun$MatrixRunExecution.getParentWorkspaceLease(MatrixRun.java:173)
	at hudson.matrix.MatrixRun$MatrixRunExecution.decideWorkspace(MatrixRun.java:187)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:509)
	at hudson.model.Run.execute(Run.java:1897)
	at hudson.matrix.MatrixRun.run(MatrixRun.java:153)

Reason Cannot read file /home/iojs/.config/jgit/config
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47035/, https://github.com/nodejs/node/pull/47055/)
Appeared test-osuosl-aix72-ppc64_be-2
First CI https://ci.nodejs.org/job/node-test-pull-request/50353/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50355/
Example
java.io.IOException: Cannot read file /home/iojs/.config/jgit/config
org.eclipse.jgit.errors.ConfigInvalidException: Invalid line in config file
	at org.eclipse.jgit.lib.Config.fromTextRecurse(Config.java:1206)
	at org.eclipse.jgit.lib.Config.fromText(Config.java:1134)
	at org.eclipse.jgit.storage.file.FileBasedConfig.lambda$0(FileBasedConfig.java:138)
	at org.eclipse.jgit.util.FileUtils.readWithRetries(FileUtils.java:722)

Git Failure
Reason ERROR: Error fetching remote repo 'origin'
Type GIT_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47132/, https://github.com/nodejs/node/pull/47091/)
Appeared test-osuosl-aix72-ppc64_be-4, test-orka-macos10.15-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50417/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50423/
Example
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git@github.com:nodejs/node.git
	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1003)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1245)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1309)
	at hudson.scm.SCM.checkout(SCM.java:540)

Build Failure
Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/47039/, https://github.com/nodejs/node/pull/47132/, https://github.com/nodejs/node/pull/47139/, https://github.com/nodejs/node/pull/47141/)
Appeared test-equinix-ubuntu2004_container-arm64-2, test-orka-macos11-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50366/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50431/
Example
fatal: No rebase in progress?

Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47068/, https://github.com/nodejs/node/pull/46815/)
Appeared test-rackspace-win2012r2_vs2017-x64-1, test-rackspace-win2012r2_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50358/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50437/
Example
error: remote GitHub already exists.

> git fetch --prune --no-tags GitHub 
From https://github.com/nodejs/build
 - [deleted]         (none)     -> GitHub/dependabot/github_actions/actions/checkout-3
   90e54ba..39694e4  main       -> GitHub/main

JSTest Failure
Reason parallel/test-child-process-exec-cwd
Type JS_TEST_FAILURE
Failed PR 13 (https://github.com/nodejs/node/pull/47108/, https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/47039/, https://github.com/nodejs/node/pull/47135/, https://github.com/nodejs/node/pull/47131/, https://github.com/nodejs/node/pull/47141/, https://github.com/nodejs/node/pull/46965/, https://github.com/nodejs/node/pull/46930/, https://github.com/nodejs/node/pull/45671/, https://github.com/nodejs/node/pull/47132/, https://github.com/nodejs/node/pull/46305/, https://github.com/nodejs/node/pull/47091/)
Appeared test-azure_msft-win11_vs2019-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50398/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50449/
Example
not ok 105 parallel/test-child-process-exec-cwd
  ---
  duration_ms: 0.322
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:399
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(stdout.startsWith(dir))
    
        at d:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-cwd.js:38:3
        at d:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:414:17
        at d:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:449:15
        at ChildProcess.exithandler (node:child_process:411:7)
        at ChildProcess.emit (node:events:510:28)
        at maybeClose (node:internal/child_process:1098:16)
        at Socket.<anonymous> (node:internal/child_process:456:11)
        at Socket.emit (node:events:510:28)
        at Pipe.<anonymous> (node:net:332:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: ...

Reason sequential/test-http-max-sockets
Type JS_TEST_FAILURE
Failed PR 11 (https://github.com/nodejs/node/pull/47108/, https://github.com/nodejs/node/pull/46979/, https://github.com/nodejs/node/pull/47091/, https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/47039/, https://github.com/nodejs/node/pull/47135/, https://github.com/nodejs/node/pull/47131/, https://github.com/nodejs/node/pull/46965/, https://github.com/nodejs/node/pull/46930/, https://github.com/nodejs/node/pull/45671/, https://github.com/nodejs/node/pull/47132/)
Appeared test-rackspace-win2012r2_vs2015-x64-2, test-rackspace-win2012r2_vs2013-x64-1, test-rackspace-win2012r2_vs2017-x64-3, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/50398/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50445/
Example
not ok 910 sequential/test-http-max-sockets
  ---
  duration_ms: 0.239
  severity: fail
  exitcode: 3221226505
  stack: |-
    Socket: 1/10 queued: 0
    Socket: 2/10 queued: 0
    Socket: 3/10 queued: 0
    Socket: 4/10 queued: 0
    Socket: 5/10 queued: 0
    Socket: 6/10 queued: 0
    Socket: 7/10 queued: 0
    Socket: 8/10 queued: 0
    Socket: 9/10 queued: 0
    Socket: 10/10 queued: 0
    Socket: 10/10 queued: 1
    Socket: 10/10 queued: 2
    Socket: 10/10 queued: 3
    Socket: 10/10 queued: 4
    Socket: 10/10 queued: 5
    Socket: 10/10 queued: 6
    Socket: 10/10 queued: 7
    Socket: 10/10 queued: 8
    Socket: 10/10 queued: 9
    Socket: 10/10 queued: 10
  ...


Reason parallel/test-http-remove-connection-header-persists-connection
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/47068/, https://github.com/nodejs/node/pull/47083/, https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/47134/, https://github.com/nodejs/node/pull/46319/)
Appeared test-digitalocean-freebsd12-x64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50358/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50430/
Example
not ok 1299 parallel/test-http-remove-connection-header-persists-connection
  ---
  duration_ms: 0.482
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at /usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
        at Timeout._onTimeout (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
        at listOnTimeout (node:internal/timers:573:17)
        at process.processTimers (node:internal/timers:514:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason pummel/test-net-throttle
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/46930/, https://github.com/nodejs/node/pull/47135/, https://github.com/nodejs/node/pull/47132/, https://github.com/nodejs/node/pull/47139/, https://github.com/nodejs/node/pull/46989/)
Appeared test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50394/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50443/
Example
not ok 3555 pummel/test-net-throttle
  ---
  duration_ms: 0.769
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
        at process.emit (node:events:522:35) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason parallel/test-http-remove-header-stays-removed
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/46692/, https://github.com/nodejs/node/pull/41439/, https://github.com/nodejs/node/pull/47139/, https://github.com/nodejs/node/pull/47039/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-softlayer-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-softlayer-ubuntu1804_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/50372/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50438/
Example
not ok 1374 parallel/test-http-remove-header-stays-removed
  ---
  duration_ms: 0.383
  severity: fail
  exitcode: 1
  stack: |-
    ok
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-http-remove-header-stays-removed.js:62:14)
        at listOnTimeout (node:internal/timers:573:17)
        at process.processTimers (node:internal/timers:514:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason pummel/test-policy-integrity-parent-no-package-json
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47079/, https://github.com/nodejs/node/pull/46815/, https://github.com/nodejs/node/pull/47033/)
Appeared test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/50377/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50446/
Example
not ok 3560 pummel/test-policy-integrity-parent-no-package-json
  ---
  duration_ms: 1.900
  severity: crashed
  exitcode: -10
  stack: |-
  ...


Reason parallel/test-child-process-bad-stdio
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47079/, https://github.com/nodejs/node/pull/47033/)
Appeared test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/50397/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50446/
Example
not ok 138 parallel/test-child-process-bad-stdio
  ---
  duration_ms: 0.85
  severity: crashed
  exitcode: -10
  stack: |-
  ...


Reason parallel/test-fs-promises-file-handle-readFile
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46930/, https://github.com/nodejs/node/pull/47131/)
Appeared test-rackspace-win2012r2_vs2019-x64-6
First CI https://ci.nodejs.org/job/node-test-pull-request/50394/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50410/
Example
not ok 257 parallel/test-fs-promises-file-handle-readFile
  ---
  duration_ms: 0.215
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/process/promises:289
                triggerUncaughtException(err, true /* fromPromise */);
                ^
    
    [Error: ENOSPC: no space left on device, ftruncate] {
      errno: -4055,
      code: 'ENOSPC',
      syscall: 'ftruncate'
    }
    
    Node.js v20.0.0-pre
  ...


Reason parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47131/, https://github.com/nodejs/node/pull/47134/)
Appeared test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/50410/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50416/
Example
not ok 1341 parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
  ---
  duration_ms: 120.34
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-fs-watch-recursive-symlink
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47139/, https://github.com/nodejs/node/pull/47091/)
Appeared test-nearform-macos11.0-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/50421/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50429/
Example
not ok 3656 parallel/test-fs-watch-recursive-symlink
  ---
  duration_ms: 360.71
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-tls-ticket-cluster
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47063/, https://github.com/nodejs/node/pull/47039/)
Appeared test-rackspace-win2012r2_vs2019-x64-4, test-azure_msft-win10_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/50354/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50413/
Example
not ok 750 parallel/test-tls-ticket-cluster
  ---
  duration_ms: 0.863
  severity: fail
  exitcode: 1
  stack: |-
    [primary] got "listening"
    [primary] connecting 61796 session? false
    [primary] got "listening"
    [primary] got "listening"
    [primary] got "listening"
    [worker] connection reused? false
    [primary] got "not-reused"
    [primary] connecting 61796 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 61796 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 61796 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 61796 session? true
    [worker] connection reused? true
    [primary] connecting 61796 session? true
    [primary] got "reused"
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 61796 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connect...

Reason parallel/test-wasm-web-api
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47108/, https://github.com/nodejs/node/pull/46815/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/50388/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50437/
Example
not ok 3801 parallel/test-wasm-web-api
  ---
  duration_ms: 960.46
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason sequential/test-inspector-debug-brk-flag
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/46930/)
Appeared test-softlayer-ubi81_container-x64-1, test-macstadium-macos11.0-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/50365/
Last CI https://ci.nodejs.org/job/node-test-pull-request/50435/
Example
not ok 3914 sequential/test-inspector-debug-brk-flag
  ---
  duration_ms: 15.344
  severity: fail
  exitcode: 1
  stack: |-
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:9229/81fa1ef5-c715-46bf-aad3-d7a41fe37f47
    [err] 
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
    Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-linux-containered/test/fixtures/loop.js:0)
    1
  ...


Progress
  • Build timed out (after 60 minutes). Marking the build as failed. (5)
  • Cannot retrieve the node's workspace for hudson.matrix.MatrixProject@5b15074c[node-test-commit-osx]. Most probably the node is offline (4)
  • Cannot read file /home/iojs/.config/jgit/config (2)
  • ERROR: Error fetching remote repo 'origin' (2)
  • fatal: No rebase in progress? (4)
  • error: remote GitHub already exists. (2)
  • parallel/test-child-process-exec-cwd (13)
  • sequential/test-http-max-sockets (11)
  • parallel/test-http-remove-connection-header-persists-connection (5)
  • pummel/test-net-throttle (5)
  • #529
  • pummel/test-policy-integrity-parent-no-package-json (3)
  • parallel/test-child-process-bad-stdio (2)
  • parallel/test-fs-promises-file-handle-readFile (2)
  • parallel/test-fs-watch-recursive-add-file-to-existing-subfolder (2)
  • parallel/test-fs-watch-recursive-symlink (2)
  • parallel/test-tls-ticket-cluster (2)
  • parallel/test-wasm-web-api (2)
  • sequential/test-inspector-debug-brk-flag (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ncu-ci report and its linked Jenkins logs, then inspect the named failures such as test-child-process-exec-cwd and test-http-max-sockets. The issue does not identify a target file or a single expected change; completion would require isolating a recurring CI failure and defining a verified fix for it.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.