nodejs / nodejs/reliability

CI Reliability 2024-03-09

Open
#806 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/57538 to node-test-pull-request/57637 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-03-09 00:07 0 0 26 8 66 0.00%
JSTest Failure
Reason parallel/test-domain-error-types
Type JS_TEST_FAILURE
Failed PR 11 (https://github.com/nodejs/node/pull/51912/, https://github.com/nodejs/node/pull/51977/, https://github.com/nodejs/node/pull/51463/, https://github.com/nodejs/node/pull/51998/, https://github.com/nodejs/node/pull/51999/, https://github.com/nodejs/node/pull/51996/, https://github.com/nodejs/node/pull/52000/, https://github.com/nodejs/node/pull/51880/, https://github.com/nodejs/node/pull/52005/, https://github.com/nodejs/node/pull/51992/, https://github.com/nodejs/node/pull/52016/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-ubuntu2004_sharedlibs_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57572/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57637/
Example
not ok 950 parallel/test-domain-error-types
  ---
  duration_ms: 1645.06200
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-child-process-exec-encoding
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51783/, https://github.com/nodejs/node/pull/51948/, https://github.com/nodejs/node/pull/51719/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57557/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57612/
Example
not ok 99 parallel/test-child-process-exec-encoding
  ---
  duration_ms: 774.03600
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:1000
        throw newErr;
        ^
    
    AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Command failed: "C:\workspace\node-test-binary-windows-js-suites\node\Release\node.exe" "C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-encoding.js" child
    
        at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:436:12
        at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:473:15
        at ChildProcess.exithandler (node:child_process:430:5)
        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at ChildProcess._handle.onexit...

Reason parallel/test-child-process-exec-maxbuf
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51783/, https://github.com/nodejs/node/pull/51948/, https://github.com/nodejs/node/pull/51719/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57557/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57612/
Example
not ok 98 parallel/test-child-process-exec-maxbuf
  ---
  duration_ms: 657.01400
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/child_process:421
        throw new ErrnoException(err, 'spawn');
        ^
    
    Error: spawn UNKNOWN
        at ChildProcess.spawn (node:internal/child_process:421:11)
        at spawn (node:child_process:761:9)
        at Object.execFile (node:child_process:351:17)
        at Object.exec (node:child_process:234:25)
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-maxbuf.js:63:6)
        at Module._compile (node:internal/modules/cjs/loader:1368:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1426:10)
        at Module.load (node:internal/modules/cjs/loader:1205:32)
        at Module._load (node:internal/modules/cjs/loader:1021:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12) {
      errno: -4094,
      code: 'UNKNOWN',
 ...

Reason parallel/test-cluster-fork-stdio
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51977/, https://github.com/nodejs/node/pull/51996/, https://github.com/nodejs/node/pull/52002/)
Appeared test-rackspace-win2016_vs2015-x64-1, test-rackspace-win2019_vs2019-x64-4, test-azure_msft-win11_vs2022-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57603/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57616/
Example
not ok 919 parallel/test-cluster-fork-stdio
  ---
  duration_ms: 120225.48000
  severity: fail
  exitcode: 1
  stack: |-
    timeout
  ...


Reason parallel/test-http-chunk-problem
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51657/, https://github.com/nodejs/node/pull/51589/, https://github.com/nodejs/node/pull/51977/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-digitalocean-ubuntu2204_sharedlibs_container-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57556/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57603/
Example
not ok 1671 parallel/test-http-chunk-problem
  ---
  duration_ms: 120028.58100
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-blob-file-backed
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51952/, https://github.com/nodejs/node/pull/52016/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-2, test-osuosl-aix72-ppc64_be-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57569/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57637/
Example
not ok 354 parallel/test-blob-file-backed
  ---
  duration_ms: 221.45300
  severity: fail
  exitcode: 1
  stack: |-
    
    node:internal/blob:291
              const error = lazyDOMException('The blob could not be read', 'NotReadableError');
                            ^
    DOMException [NotReadableError]: The blob could not be read
        at BlobReader.<anonymous> (node:internal/blob:291:25)
    
    Node.js v22.0.0-pre
  ...


Reason parallel/test-buffer-constructor-node-modules-paths
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51783/, https://github.com/nodejs/node/pull/51948/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57557/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57558/
Example
not ok 87 parallel/test-buffer-constructor-node-modules-paths
  ---
  duration_ms: 329.05500
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    + actual - expected
    
    + '<--- Last few GCs --->\r\n' +
    +   '\r\n' +
    +   '\r\n' +
    +   '<--- JS stacktrace --->\r\n' +
    +   '\r\n' +
    +   '\r\n' +
    +   '#\r\n' +
    +   '# Fatal JavaScript out of memory: MemoryChunk allocation failed during deserialization.\r\n' +
    +   '#'
    - ''
        at test (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-buffer-constructor-node-modules-paths.js:20:12)
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-buffer-constructor-node-modules-paths.js:23:1)
        at Module._compile (node:internal/modules/cjs/loader:1368:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:14...

Reason parallel/test-child-process-exec-timeout-kill
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51783/, https://github.com/nodejs/node/pull/51719/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57557/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57612/
Example
not ok 100 parallel/test-child-process-exec-timeout-kill
  ---
  duration_ms: 504.03200
  severity: fail
  exitcode: 134
  stack: |-
    
    <--- Last few GCs --->
    
    
    <--- JS stacktrace --->
    
    FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory
    ----- Native stack trace -----
    
     1: 00007FF7D0D8FD3B 
     2: 00007FF7D0CF9844 
     3: 00007FF7D18BC711 
     4: 00007FF7D18A5E18 
     5: 00007FF7D16F3980 
     6: 00007FF7D16ACF93 
     7: 00007FF7D16FC855 
     8: 00007FF7D16ECFD2 
     9: 00007FF7D16E8F45 
    10: 00007FF7D10228FD 
  ...


Reason parallel/test-fs-read-stream-concurrent-reads
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51884/, https://github.com/nodejs/node/pull/51977/)
Appeared test-equinix-ubuntu2004_container-armv7l-1, test-equinix-debian11_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57541/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57594/
Example
not ok 3765 parallel/test-fs-read-stream-concurrent-reads
  ---
  duration_ms: 360036.66200
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51743/, https://github.com/nodejs/node/pull/51972/)
Appeared test-digitalocean-freebsd12-x64-1, test-digitalocean-fedora32-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57566/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57592/
Example
not ok 2401 parallel/test-runner-output
  ---
  duration_ms: 32019.22800
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 28175.63798
          ...


Reason parallel/test-worker-arraybuffer-zerofill
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51362/, https://github.com/nodejs/node/pull/52005/)
Appeared test-equinix_mnx-smartos20-x64-3, test-digitalocean-ubuntu2204-x64-2, test-ibm-ubuntu2204_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/57546/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57621/
Example
not ok 3723 parallel/test-worker-arraybuffer-zerofill
  ---
  duration_ms: 300107.52500
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason pummel/test-crypto-timing-safe-equal-benchmarks
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51948/, https://github.com/nodejs/node/pull/51932/)
Appeared test-macstadium-macos11.0-arm64-4, test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57559/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57596/
Example
not ok 3712 pummel/test-crypto-timing-safe-equal-benchmarks
  ---
  duration_ms: 1654.64400
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:408
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: timingSafeEqual should not leak information from its execution time (t=-3.902590470192131)
        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/pummel/test-crypto-timing-safe-equal-benchmarks.js:109:1)
        at Module._compile (node:internal/modules/cjs/loader:1368:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1426:10)
        at Module.load (node:internal/modules/cjs/loader:1205:32)
        at Module._load (node:internal/modules/cjs/loader:1021:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12)
        at node:internal/main/run_main_module:28:49 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '...

Reason sequential/test-tls-session-timeout
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51977/, https://github.com/nodejs/node/pull/51996/)
Appeared test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57603/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57608/
Example
not ok 956 sequential/test-tls-session-timeout
  ---
  duration_ms: 2480.32100
  severity: fail
  exitcode: 1
  stack: |-
    C:\workspace\node-test-binary-windows-js-suites\node\test\sequential\test-tls-session-timeout.js:114
            throw er;
            ^
    
    [Error: 28110000:error:0A00041A:SSL routines:ssl3_read_bytes:tlsv1 alert decode error:C:\workspace\node-compile-windows\node\deps\openssl\openssl\ssl\record\rec_layer_s3.c:1590:SSL alert number 50
    ] {
      library: 'SSL routines',
      reason: 'tlsv1 alert decode error',
      code: 'ERR_SSL_TLSV1_ALERT_DECODE_ERROR'
    }
    
    Node.js v20.11.2-pre
  ...


Reason sequential/test-worker-fshandles-open-close-on-termination
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51719/, https://github.com/nodejs/node/pull/52002/)
Appeared test-rackspace-win2022_vs2022-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57612/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57616/
Example
not ok 959 sequential/test-worker-fshandles-open-close-on-termination
  ---
  duration_ms: 1063.02700
  severity: crashed
  exitcode: 3221225477
  stack: |-
  ...


Jenkins Failure
Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51463/, https://github.com/nodejs/node/pull/51973/)
Appeared test-digitalocean-freebsd12-x64-2, test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57595/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57602/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 107.170.28.213/107.170.28.213:12277' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
	at jdk.proxy2/jdk.proxy2.$Proxy186.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)

Reason Build timed out (after 120 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51903/, https://github.com/nodejs/node/pull/52016/)
Appeared test-digitalocean-fedora32-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57583/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57637/
Example
Build timed out (after 120 minutes). Marking the build as failed.

Git Failure
Build Failure
Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/51362/, https://github.com/nodejs/node/pull/51932/, https://github.com/nodejs/node/pull/51463/, https://github.com/nodejs/node/pull/52002/, https://github.com/nodejs/node/pull/52016/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-9
First CI https://ci.nodejs.org/job/node-test-pull-request/57546/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57637/
Example
fatal: No rebase in progress?

Reason fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)
Type BUILD_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/51880/, https://github.com/nodejs/node/pull/51903/, https://github.com/nodejs/node/pull/51932/, https://github.com/nodejs/node/pull/51973/)
Appeared test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57578/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57591/
Example
fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device)

Reason fatal error: error in backend: IO failure on output stream: No space left on device
Type BUILD_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/51880/, https://github.com/nodejs/node/pull/51932/, https://github.com/nodejs/node/pull/51976/, https://github.com/nodejs/node/pull/51977/)
Appeared test-macstadium-macos11.0-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57587/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57594/
Example
fatal error: error in backend: IO failure on output stream: No space left on device

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51820/, https://github.com/nodejs/node/pull/51589/)
Appeared test-equinix-debian11_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-1, test-equinix-debian11_container-armv7l-2, test-osuosl-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57579/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57586/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
[PostBuildScript] - [INFO] Executing post build scripts.
[node-test-binary-armv7l] $ /bin/bash -ex /tmp/jenkins7233274798554556647.sh
+ '[' -d .git ']'
+ git clean -fdx
Removing out/

CCTest Failure
Reason Value of: (!expectation.read_expected)
Type CC_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51943/, https://github.com/nodejs/node/pull/51903/, https://github.com/nodejs/node/pull/51998/)
Appeared test-digitalocean-alpine318_container-x64-1, test-digitalocean-alpine315_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57552/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57604/
Example
Value of: (!expectation.read_expected)
  Actual: true
Expected: false

[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (4999 ms)

Progress
  • parallel/test-domain-error-types (11)
  • parallel/test-child-process-exec-encoding (3)
  • parallel/test-child-process-exec-maxbuf (3)
  • parallel/test-cluster-fork-stdio (3)
  • parallel/test-http-chunk-problem (3)
  • parallel/test-blob-file-backed (2)
  • parallel/test-buffer-constructor-node-modules-paths (2)
  • parallel/test-child-process-exec-timeout-kill (2)
  • parallel/test-fs-read-stream-concurrent-reads (2)
  • parallel/test-runner-output (2)
  • parallel/test-worker-arraybuffer-zerofill (2)
  • pummel/test-crypto-timing-safe-equal-benchmarks (2)
  • sequential/test-tls-session-timeout (2)
  • sequential/test-worker-fshandles-open-close-on-termination (2)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (2)
  • Build timed out (after 120 minutes). Marking the build as failed. (2)
  • fatal: No rebase in progress? (5)
  • fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write to output file (No space left on device) (4)
  • fatal error: error in backend: IO failure on output stream: No space left on device (4)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (2)
  • Value of: (!expectation.read_expected) (3)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked CI console logs for the listed failures, especially parallel/test-domain-error-types and the Windows child-process tests, then compare their first and last CI runs. Trace each failure across the named test files and platforms; done means identifying and resolving the reported reliability failures so the affected CI runs pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.