nodejs / nodejs/reliability

CI Reliability 2024-06-21

Open
#902 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/59807 to node-test-pull-request/59898 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-06-21 00:08 0 13 18 2 67 13.27%
Jenkins Failure
Reason Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory
Type JENKINS_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/53465/, https://github.com/nodejs/node/pull/53468/, https://github.com/nodejs/node/pull/53109/, https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53495/)
Appeared test-ibm-rhel9-x64-1, test-digitalocean-fedora39-x64-1, test-digitalocean-fedora38-x64-2, test-digitalocean-fedora38-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59847/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59862/
Example
java.io.IOException: Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
	at hudson.Proc$LocalProc.<init>(Proc.java:252)
	at hudson.Proc$LocalProc.<init>(Proc.java:221)
	at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)

Reason Build timed out (after 30 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53107/, https://github.com/nodejs/node/pull/53511/, https://github.com/nodejs/node/pull/53239/)
Appeared test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-1, test-rackspace-win2019_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59898/
Example
Build timed out (after 30 minutes). Marking the build as failed.

Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/53475/)
Appeared test-digitalocean-fedora38-x64-2, test-rackspace-win2022_vs2022-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59843/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59864/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 134.209.172.40/134.209.172.40:59752' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
	at jdk.proxy2/jdk.proxy2.$Proxy103.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)

Git Failure
Build Failure
Reason fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)
Type BUILD_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/53411/, https://github.com/nodejs/node/pull/53488/, https://github.com/nodejs/node/pull/53468/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53107/)
Appeared test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59839/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Example
fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)

Reason Error: invalid syntax
Type BUILD_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/53462/, https://github.com/nodejs/node/pull/53465/, https://github.com/nodejs/node/pull/53411/, https://github.com/nodejs/node/pull/53488/, https://github.com/nodejs/node/pull/53486/)
Appeared test-orka-macos10.15-x64-2, test-rackspace-debian10-x64-1, test-orka-macos10.15-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59835/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59841/
Example
Error: invalid syntax
make: *** [build-ci] Error 1
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/53467/, https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53495/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-3, test-ibm-ubuntu2204_sharedlibs_container-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59829/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59878/
Example
fatal: No rebase in progress?

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53477/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53503/)
Appeared test-mnx-smartos21-x64-2, test-mnx-smartos23-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59824/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59870/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

JSTest Failure
Reason sequential/test-http-server-request-timeouts-mixed
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/53467/, https://github.com/nodejs/node/pull/53465/, https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53107/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53411/)
Appeared test-orka-macos11-x64-1, test-orka-macos11-x64-2, test-orka-macos10.15-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59813/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59894/
Example
not ok 4118 sequential/test-http-server-request-timeouts-mixed
  ---
  duration_ms: 3135.54400
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:408
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(request2.completed)
    
        at Timeout._onTimeout (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/sequential/test-http-server-request-timeouts-mixed.js:108:5)
        at listOnTimeout (node:internal/timers:581:17)
        at process.processTimers (node:internal/timers:519:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v23.0.0-pre
  ...


Reason benchmark/test-benchmark-crypto
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/53488/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/44340/)
Appeared test-osuosl-aix72-ppc64_be-3, test-digitalocean-ubuntu2204_sharedlibs_container-x64-3, test-ibm-ubuntu2204-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/59840/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59887/
Example
not ok 4024 benchmark/test-benchmark-crypto
  ---
  duration_ms: 5666.56300
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== 0
    
        at main (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/benchmark/crypto/timingSafeEqual.js:21:10)
        at /home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/benchmark/common.js:54:9
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: 0,
      operator: 'strictEqual'
    }
    
    Node.js v23.0.0-pre
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== 0
    
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72...

Reason parallel/test-cli-node-options
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53465/)
Appeared test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2019_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/59842/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59847/
Example
not ok 121 parallel/test-cli-node-options
  ---
  duration_ms: 1341.04800
  severity: crashed
  exitcode: 2147483651
  stack: |-
    
    <--- Last few GCs --->
    
    
    <--- JS stacktrace --->
    
    FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
    ----- Native stack trace -----
    
    
    #
    # Fatal javascript OOM in MemoryChunk allocation failed during deserialization.
    #
    
  ...


Reason parallel/test-pipe-file-to-http
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/53464/, https://github.com/nodejs/node/pull/53472/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53107/)
Appeared test-equinix-debian11_container-armv7l-2, test-osuosl-debian11_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-1, test-equinix-debian11_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59811/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Example
not ok 3860 parallel/test-pipe-file-to-http
  ---
  duration_ms: 360064.57300
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-require-symlink
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53465/)
Appeared test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59842/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59847/
Example
not ok 660 parallel/test-require-symlink
  ---
  duration_ms: 429.00500
  severity: crashed
  exitcode: 3221225477
  stack: |-
  ...


Reason sequential/test-module-loading
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/44340/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53107/)
Appeared test-rackspace-win2016_vs2015-x64-1, test-azure_msft-win11_vs2022-arm64-2, test-orka-macos10.15-x64-1, test-azure_msft-win10_vs2019-x64-1, test-azure_msft-win11_vs2022-x64-1, test-azure_msft-win2016_vs2017-x64-2, test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-3, test-digitalocean-alpine318_container-x64-1, test-rackspace-debian10-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59872/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Example
not ok 969 sequential/test-module-loading
  ---
  duration_ms: 226.99900
  severity: fail
  exitcode: 1
  stack: |-
    b/c.js exit
    load test-module-loading.js
    load fixtures/b/d.js
    load package/index.js
    load fixtures/b/c.js
    load fixtures/a.js
    test index.js modules ids and relative loading
    test index.js in a folder with a trailing slash
    node:assert:758
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^SyntaxError: Error parsing/. Input:
    
    'Error: Invalid package config C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\fixtures\\packages\\unparseable\\package.json.'
    
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-module-loading.js:129:8)
        at Module._compile (node:internal/modules/cjs/loader:1357:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1415:10)
        at Module.load (node:internal/modules/cjs/...

Reason parallel/test-file-write-stream4
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53468/, https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/44340/)
Appeared test-ibm-alpine319_container-x64-1, test-equinix-rhel8_container-arm64-1, test-equinix-ubuntu2004_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59849/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59887/
Example
not ok 1712 parallel/test-file-write-stream4
  ---
  duration_ms: 120037.00100
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53431/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/44340/)
Appeared test-ibm-rhel8-x64-1, test-ibm-ubuntu2204-x64-2, test-ibm-rhel8-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59826/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59887/
Example
not ok 2397 parallel/test-performance-function
  ---
  duration_ms: 275.52700
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/histogram:291
        validateInteger(val, 'val', 1);
        ^
    
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:291:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-performance-function.js:103:5
        at runNextTicks (node:internal/process/task_queues:60:5)
        at listOnTimeout (node:internal/timers:545:9)
        at process.processTimers (node:internal/timers:519:7) {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v23.0.0-pre
  ...


Reason parallel/test-domain-error-types
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53109/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/59848/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59856/
Example
not ok 717 parallel/test-domain-error-types
  ---
  duration_ms: 2230.05900
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53500/, https://github.com/nodejs/node/pull/53511/)
Appeared test-osuosl-aix72-ppc64_be-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59866/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59892/
Example
not ok 2706 parallel/test-runner-output
  ---
  duration_ms: 4372.16000
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 2891.149953
          ...


Progress
  • Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory (8)
  • Build timed out (after 30 minutes). Marking the build as failed. (3)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (2)
  • fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device) (10)
  • Error: invalid syntax (5)
  • fatal: No rebase in progress? (5)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (3)
  • sequential/test-http-server-request-timeouts-mixed (6)
  • benchmark/test-benchmark-crypto (4)
  • parallel/test-cli-node-options (4)
  • parallel/test-pipe-file-to-http (4)
  • parallel/test-require-symlink (4)
  • sequential/test-module-loading (4)
  • parallel/test-file-write-stream4 (3)
  • parallel/test-performance-function (3)
  • parallel/test-domain-error-types (2)
  • parallel/test-runner-output (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ncu-ci walk pr --stats=true --markdown .../reliability/results.md report and inspect the linked Jenkins console logs for the recurring failure categories. Compare the affected node-test-pull-request jobs and platform-specific jobs; done means the repeated CI failures have an identified resolution and the reliability report no longer shows them recurring.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.