nodejs / nodejs/reliability

CI Reliability 2024-06-25

Open
#906 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/59858 to node-test-pull-request/59957 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-06-25 00:08 0 8 25 5 62 8.42%
JSTest Failure
Reason parallel/test-snapshot-reproducible
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/53239/, https://github.com/nodejs/node/pull/53488/, https://github.com/nodejs/node/pull/52206/, https://github.com/nodejs/node/pull/53537/, https://github.com/nodejs/node/pull/53044/, https://github.com/nodejs/node/pull/52908/, https://github.com/nodejs/node/pull/53563/)
Appeared test-digitalocean-ubuntu2204_sharedlibs_container-x64-2, test-ibm-ubuntu2204_sharedlibs_container-x64-5, test-digitalocean-ubuntu2204_sharedlibs_container-x64-7, test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-3, test-digitalocean-ubuntu2204_sharedlibs_container-x64-5, test-digitalocean-ubuntu2204_sharedlibs_container-x64-8, test-digitalocean-ubuntu2204_sharedlibs_container-x64-6, test-digitalocean-ubuntu2204_sharedlibs_container-x64-10, test-ibm-rhel8-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59898/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59957/
Example
not ok 2498 parallel/test-snapshot-reproducible
  ---
  duration_ms: 1435.69600
  severity: fail
  exitcode: 1
  stack: |-
    0x0: Write magic 143da19
    0x4: Write metadata
    0x37: Write snapshot blob
    0x17328b: Write IsolateDataSerializeInfo
    0x174767: Write EnvSerializeInfo
    0x175600: Write CodeCacheInfo
    0x0: Write magic 143da19
    0x4: Write metadata
    0x37: Write snapshot blob
    0x17328b: Write IsolateDataSerializeInfo
    0x174767: Write EnvSerializeInfo
    0x175600: Write CodeCacheInfo
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
    + actual - expected
    
    + [
    +   {
    +     offset: '0x40',
    +     slice1: '00000074bfb266b8eea9aa31322e342e',
    +     slice2: '00000099c079e9b8eea9aa31322e342e'
    +   },
    +   {
    +     offset: '0x1015a0',
    +     slice1: '805d44660000000000000000901a1edf',
    +     slice2: '805d44660000000000000000904af7a1'
    +   },
    +...

Reason sequential/test-http-server-request-timeouts-mixed
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/53107/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53411/, https://github.com/nodejs/node/pull/53517/, https://github.com/nodejs/node/pull/53554/, https://github.com/nodejs/node/pull/53555/)
Appeared test-orka-macos11-x64-2, test-orka-macos11-x64-1, test-orka-macos10.15-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59867/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59944/
Example
not ok 4124 sequential/test-http-server-request-timeouts-mixed
  ---
  duration_ms: 3175.12300
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:408
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(request2.completed)
    
        at Timeout._onTimeout (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/sequential/test-http-server-request-timeouts-mixed.js:108:5)
        at listOnTimeout (node:internal/timers:581:17)
        at process.processTimers (node:internal/timers:519:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v23.0.0-pre
  ...


Reason benchmark/test-benchmark-crypto
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/44340/, https://github.com/nodejs/node/pull/53540/, https://github.com/nodejs/node/pull/53555/)
Appeared test-ibm-alpine319_container-x64-1, test-osuosl-rhel8-ppc64_le-4, test-osuosl-aix72-ppc64_be-3, test-digitalocean-ubuntu2204_sharedlibs_container-x64-3, test-ibm-ubuntu2204-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59871/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59936/
Example
not ok 4025 benchmark/test-benchmark-crypto
  ---
  duration_ms: 3419.23600
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== 0
    
        at main (/home/iojs/build/workspace/node-test-commit-linux/benchmark/crypto/timingSafeEqual.js:21:10)
        at /home/iojs/build/workspace/node-test-commit-linux/benchmark/common.js:54:9
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: 0,
      operator: 'strictEqual'
    }
    
    Node.js v23.0.0-pre
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== 0
    
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/common/benchmark.js:28:12)
        a...

Reason parallel/test-blob-file-backed
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53550/, https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/53044/)
Appeared test-azure_msft-win11_vs2022-x64-2, test-digitalocean-fedora38-x64-2, test-digitalocean-ubuntu2204-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59870/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59945/
Example
not ok 85 parallel/test-blob-file-backed
  ---
  duration_ms: 183.00300
  severity: fail
  exitcode: 1
  stack: |-
    
    node:internal/blob:290
              const error = lazyDOMException('The blob could not be read', 'NotReadableError');
                            ^
    DOMException [NotReadableError]: The blob could not be read
        at BlobReader.<anonymous> (node:internal/blob:290:25)
    
    Node.js v23.0.0-pre
  ...


Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/44340/, https://github.com/nodejs/node/pull/53517/, https://github.com/nodejs/node/pull/52881/, https://github.com/nodejs/node/pull/53457/)
Appeared undefined, test-ibm-ubuntu2204-x64-2, test-ibm-rhel8-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59887/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59954/
Example
not ok 2452 parallel/test-performance-function
  ---
  duration_ms: 411.06100
  severity: fail
  exitcode: 1
  stack: |-
    
    node:internal/event_target:1090
      process.nextTick(() => { throw err; });
                               ^
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:291:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/parallel/test-performance-function.js:103:5 {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v23.0.0-pre
  ...


Reason sequential/test-module-loading
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/44340/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53107/)
Appeared test-rackspace-win2016_vs2015-x64-1, test-azure_msft-win11_vs2022-arm64-2, test-orka-macos10.15-x64-1, test-azure_msft-win10_vs2019-x64-1, test-azure_msft-win11_vs2022-x64-1, test-azure_msft-win2016_vs2017-x64-2, test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-3, test-digitalocean-alpine318_container-x64-1, test-rackspace-debian10-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59872/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Example
not ok 969 sequential/test-module-loading
  ---
  duration_ms: 226.99900
  severity: fail
  exitcode: 1
  stack: |-
    b/c.js exit
    load test-module-loading.js
    load fixtures/b/d.js
    load package/index.js
    load fixtures/b/c.js
    load fixtures/a.js
    test index.js modules ids and relative loading
    test index.js in a folder with a trailing slash
    node:assert:758
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^SyntaxError: Error parsing/. Input:
    
    'Error: Invalid package config C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\fixtures\\packages\\unparseable\\package.json.'
    
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-module-loading.js:129:8)
        at Module._compile (node:internal/modules/cjs/loader:1357:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1415:10)
        at Module.load (node:internal/modules/cjs/...

Reason parallel/test-file-write-stream4
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/44340/)
Appeared test-ibm-alpine319_container-x64-1, test-equinix-rhel8_container-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59865/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59887/
Example
not ok 1712 parallel/test-file-write-stream4
  ---
  duration_ms: 120037.00100
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-http2-large-write-multiple-requests
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53539/, https://github.com/nodejs/node/pull/53558/)
Appeared test-ibm-alpine319_container-x64-1, test-rackspace-win2022_vs2022-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59917/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59940/
Example
not ok 2786 parallel/test-http2-large-write-multiple-requests
  ---
  duration_ms: 120038.96600
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    server sends content 1
    server sends content 2
    server sends content 3
    server sends content 4
    server sends content 5
    server sends content 6
    server sends content 7
    server sends content 8
    server sends content 9
    server sends content 10
    server sends content 11
    server sends content 12
    server sends content 13
    server sends content 14
    server sends content 15
    server sends content 16
    server sends content 17
    server sends content 18
    server sends content 19
    server sends content 20
    server sends content 21
    server sends content 22
    server sends content 23
    server sends content 24
    server sends content 25
    server sends content 26
    server sends content 27
    server sends content 28
    server sends content 29
    server sends content 30
    server sends content 31
    server s...

Reason parallel/test-net-write-fully-async-buffer
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53488/)
Appeared test-equinix_mnx-smartos20-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59870/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59925/
Example
not ok 3816 parallel/test-net-write-fully-async-buffer
  ---
  duration_ms: 300143.44400
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53500/, https://github.com/nodejs/node/pull/53511/)
Appeared test-osuosl-aix72-ppc64_be-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59866/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59892/
Example
not ok 2706 parallel/test-runner-output
  ---
  duration_ms: 4372.16000
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 2891.149953
          ...


Reason sequential/test-performance-eventloopdelay
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53540/, https://github.com/nodejs/node/pull/53545/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-3, test-ibm-ubuntu2204_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/59918/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59923/
Example
not ok 4139 sequential/test-performance-eventloopdelay
  ---
  duration_ms: 7225.76200
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:408
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(histogram.min > 0)
    
        at Timeout.spinAWhile [as _onTimeout] (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-performance-eventloopdelay.js:65:7)
        at listOnTimeout (node:internal/timers:581:17)
        at process.processTimers (node:internal/timers:519:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v23.0.0-pre
  ...


Jenkins Failure
Reason Build timed out (after 30 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/53107/, https://github.com/nodejs/node/pull/53511/, https://github.com/nodejs/node/pull/53239/, https://github.com/nodejs/node/pull/52908/, https://github.com/nodejs/node/pull/53537/, https://github.com/nodejs/node/pull/53488/)
Appeared test-rackspace-win2022_vs2022-x64-1, test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2019_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59926/
Example
Build timed out (after 30 minutes). Marking the build as failed.

Reason Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory
Type JENKINS_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/53486/, https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53495/)
Appeared test-ibm-rhel9-x64-1, test-digitalocean-fedora39-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59858/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59862/
Example
java.io.IOException: Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
	at hudson.Proc$LocalProc.<init>(Proc.java:252)
	at hudson.Proc$LocalProc.<init>(Proc.java:221)
	at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)

Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53522/)
Appeared test-rackspace-fedora32-x64-1, test-digitalocean-fedora38-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59864/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59908/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 119.9.51.79/119.9.51.79:48336' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
	at jdk.proxy2/jdk.proxy2.$Proxy195.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)

Build Failure
Reason fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)
Type BUILD_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/52135/, https://github.com/nodejs/node/pull/53063/, https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53413/, https://github.com/nodejs/node/pull/53107/)
Appeared test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/59859/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59881/
Example
fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53488/, https://github.com/nodejs/node/pull/53550/)
Appeared test-mnx-smartos21-x64-2, test-mnx-smartos23-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59870/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59928/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/53503/, https://github.com/nodejs/node/pull/53475/, https://github.com/nodejs/node/pull/53495/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59870/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59878/
Example
fatal: No rebase in progress?

Reason ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for ...
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53495/, https://github.com/nodejs/node/pull/53457/)
Appeared test-rackspace-fedora32-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59947/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59954/
Example
ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for node-test-commit-linux/nodes=fedora-last-latest-x64 #58567
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason fatal error: Killed signal terminated program cc1plus
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/53522/, https://github.com/nodejs/node/pull/52206/)
Appeared test-rackspace-fedora32-x64-1, test-digitalocean-alpine319_container-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-7, test-digitalocean-rhel8_arm_cross_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/59908/
Last CI https://ci.nodejs.org/job/node-test-pull-request/59949/
Example
fatal error: Killed signal terminated program cc1plus

Progress
  • parallel/test-snapshot-reproducible (7)
  • sequential/test-http-server-request-timeouts-mixed (7)
  • benchmark/test-benchmark-crypto (5)
  • parallel/test-blob-file-backed (4)
  • parallel/test-performance-function (4)
  • sequential/test-module-loading (4)
  • parallel/test-file-write-stream4 (2)
  • parallel/test-http2-large-write-multiple-requests (2)
  • parallel/test-net-write-fully-async-buffer (2)
  • parallel/test-runner-output (2)
  • sequential/test-performance-eventloopdelay (2)
  • Build timed out (after 30 minutes). Marking the build as failed. (6)
  • Cannot run program "/bin/sh" (in directory "/home/iojs/build/workspace/node-test-commit-linux"): error=2, No such file or directory (5)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (2)
  • fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device) (6)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (3)
  • fatal: No rebase in progress? (3)
  • ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for ... (2)
  • fatal error: Killed signal terminated program cc1plus (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the listed failures, especially test/sequential/test-http-server-request-timeouts-mixed.js, benchmark/crypto/timingSafeEqual.js, and the parallel tests, then reproduce the failures from the referenced CI logs. Determine which failures remain reproducible and address the underlying reliability problems. Done means the affected tests pass consistently across the reported environments.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.