nodejs / nodejs/reliability

CI Reliability 2024-03-15

Open
#812 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/57650 to node-test-pull-request/57748 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-03-15 00:07 0 0 26 6 68 0.00%
JSTest Failure
Reason parallel/test-domain-error-types
Type JS_TEST_FAILURE
Failed PR 12 (https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/52028/, https://github.com/nodejs/node/pull/51866/, https://github.com/nodejs/node/pull/52009/, https://github.com/nodejs/node/pull/52036/, https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/51791/, https://github.com/nodejs/node/pull/52039/, https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52037/, https://github.com/nodejs/node/pull/52038/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-4, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-ubuntu2004_sharedlibs_container-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57650/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57718/
Example
not ok 945 parallel/test-domain-error-types
  ---
  duration_ms: 1441.34600
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-debugger-heap-profiler
Type JS_TEST_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/52009/, https://github.com/nodejs/node/pull/51972/, https://github.com/nodejs/node/pull/52037/, https://github.com/nodejs/node/pull/52027/, https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/51866/, https://github.com/nodejs/node/pull/52023/, https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52040/)
Appeared test-azure_msft-win11_vs2022-arm64-4, test-azure_msft-win2016_vs2017-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57665/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57709/
Example
not ok 182 parallel/test-debugger-heap-profiler
  ---
  duration_ms: 2464.99000
  severity: fail
  exitcode: 1
  stack: |-
    Can't clean tmpdir: d:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.180
    Files blocking: [ 'node.heapsnapshot' ]
    
    d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:69
        throw e;
        ^
    
    Error: EBUSY: resource busy or locked, rmdir 'd:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.180'
        at rmdirSync (node:fs:1219:11)
        at _rmdirSync (node:internal/fs/rimraf:235:5)
        at rimrafSync (node:internal/fs/rimraf:193:7)
        at Object.rmSync (node:fs:1268:10)
        at rmSync (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:20:8)
        at onexit (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:54:5)
        at process.<anonymous> (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:43:14)
        at process.emit (...

Reason sequential/test-worker-fshandles-open-close-on-termination
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52023/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52037/)
Appeared test-rackspace-win2019_vs2019-x64-3, test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win11_vs2022-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57688/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57713/
Example
not ok 973 sequential/test-worker-fshandles-open-close-on-termination
  ---
  duration_ms: 1050.04300
  severity: crashed
  exitcode: 3221225477
  stack: |-
  ...


Reason benchmark/test-benchmark-net
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51973/, https://github.com/nodejs/node/pull/52037/)
Appeared test-azure_msft-win10_vs2019-x64-2, test-azure_msft-win11_vs2022-x64-3, test-azure_msft-win11_vs2022-arm64-4, test-rackspace-win2019_vs2019-x64-3, test-rackspace-win2022_vs2022-x64-1, test-azure_msft-win11_vs2022-x64-4, test-rackspace-win2022_vs2022-x64-2, test-azure_msft-win11_vs2022-x64-2, test-azure_msft-win11_vs2022-arm64-2, test-rackspace-win2019_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-4, test-azure_msft-win10_vs2019-x64-4, test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2022_vs2022-x64-3, test-azure_msft-win10_vs2019-x64-3, test-rackspace-win2019_vs2019-x64-2, test-rackspace-win2022_vs2022-x64-6
First CI https://ci.nodejs.org/job/node-test-pull-request/57666/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57712/
Example
not ok 195 benchmark/test-benchmark-net
  ---
  duration_ms: 1030.07000
  severity: fail
  exitcode: 1
  stack: |-
    node:events:497
          throw er; // Unhandled 'error' event
          ^
    
    Error: write ENOBUFS
        at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16)
    Emitted 'error' event on Socket instance at:
        at emitErrorNT (node:internal/streams/destroy:169:8)
        at emitErrorCloseNT (node:internal/streams/destroy:128:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      errno: -4060,
      code: 'ENOBUFS',
      syscall: 'write'
    }
    
    Node.js v22.0.0-pre
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== 0
    
        at ChildProcess.<anonymous> (c:\workspace\node-test-binary-windows-native-suites\node\test\common\benchmark.js:28:12)
        at ChildProcess.emit (node:events:519:2...

Reason parallel/test-blob-file-backed
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52037/, https://github.com/nodejs/node/pull/52062/)
Appeared test-digitalocean-alpine318_container-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/57664/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57719/
Example
not ok 78 parallel/test-blob-file-backed
  ---
  duration_ms: 437.07400
  severity: fail
  exitcode: 1
  stack: |-
    
    node:internal/blob:291
              const error = lazyDOMException('The blob could not be read', 'NotReadableError');
                            ^
    DOMException [NotReadableError]: The blob could not be read
        at BlobReader.<anonymous> (node:internal/blob:291:25)
    
    Node.js v22.0.0-pre
  ...


Reason parallel/test-child-process-exec-encoding
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52010/, https://github.com/nodejs/node/pull/52027/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57655/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57674/
Example
not ok 102 parallel/test-child-process-exec-encoding
  ---
  duration_ms: 870.05200
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:1000
        throw newErr;
        ^
    
    AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Command failed: "C:\workspace\node-test-binary-windows-js-suites\node\Release\node.exe" "C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-encoding.js" child
    
    
    #
    
    # Fatal process out of memory: Re-embedded builtins: set permissions
    
    #
    
    
    
    
        at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:436:12
        at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:473:15
        at ChildProcess.exithandler (node:child_process:430:5)
        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at ChildProcess.exithandler (node:child_process:422:12)
        at ChildProcess.emi...

Reason parallel/test-child-process-exec-maxbuf
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52010/, https://github.com/nodejs/node/pull/52027/)
Appeared test-rackspace-win2019_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57655/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57674/
Example
not ok 98 parallel/test-child-process-exec-maxbuf
  ---
  duration_ms: 634.02800
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/child_process:421
        throw new ErrnoException(err, 'spawn');
        ^
    
    Error: spawn UNKNOWN
        at ChildProcess.spawn (node:internal/child_process:421:11)
        at spawn (node:child_process:761:9)
        at Object.execFile (node:child_process:351:17)
        at Object.exec (node:child_process:234:25)
        at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-maxbuf.js:116:20)
        at Module._compile (node:internal/modules/cjs/loader:1368:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1426:10)
        at Module.load (node:internal/modules/cjs/loader:1205:32)
        at Module._load (node:internal/modules/cjs/loader:1021:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:177:12) {
      errno: -4094,
      code: 'UNKNOWN',...

Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51050/, https://github.com/nodejs/node/pull/51977/)
Appeared test-ibm-rhel8-x64-1, test-ibm-rhel8-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57736/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57748/
Example
not ok 2346 parallel/test-performance-function
  ---
  duration_ms: 195.65100
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/histogram:291
        validateInteger(val, 'val', 1);
        ^
    
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:291:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-performance-function.js:103:5 {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v22.0.0-pre
  ...


Reason parallel/test-shadow-realm-gc-module
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/51362/)
Appeared test-digitalocean-ubi81_container-x64-1, test-ibm-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57700/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57722/
Example
not ok 2431 parallel/test-shadow-realm-gc-module
  ---
  duration_ms: 5262.37100
  severity: crashed
  exitcode: -6
  stack: |-
    
    <--- Last few GCs --->
    ) [2298362:0x6515000]     4715 ms: Mark-Compact 19.1 (32.4) -> 16.5 (33.4) MB, pooled: 0 MB, 12.07 / 0.01 ms  (+ 16.1 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 43 ms) (average mu = 0.627, current mu = 0.661) [2298362:0x6515000]     4886 ms: Mark-Compact (reduce) 23.6 (34.4) -> 20.1 (21.9) MB, pooled: 0 MB, 38.68 / 0.01 ms  (+ 43.8 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 129 ms) (average mu = 0.620, current mu
    
    <--- JS stacktrace --->
    
    FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
    ----- Native stack trace -----
    
     1: 0xd70c70 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [out/Release/node]
     2: 0x10dc910 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OO...

Reason parallel/test-tls-ticket-cluster
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/52077/)
Appeared test-ibm-rhel8-x64-1, test-azure_msft-win11_vs2022-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57680/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57741/
Example
not ok 3186 parallel/test-tls-ticket-cluster
  ---
  duration_ms: 772.92600
  severity: fail
  exitcode: 1
  stack: |-
    [primary] got "listening"
    [primary] connecting 45979 session? false
    [worker] connection reused? false
    [primary] got "not-reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 45979 session? true
    [worker] connection reused? true
    [primary] got "reuse...

Reason parallel/test-worker-memory
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/52077/)
Appeared test-rackspace-win2022_vs2022-x64-6, test-azure_msft-win10_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57708/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57741/
Example
not ok 882 parallel/test-worker-memory
  ---
  duration_ms: 1807.01800
  severity: fail
  exitcode: 3221226356
  stack: |-
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=7.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=6.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8)
    run() called with n=4.5 (numWorkers=8)
    run() called with n=5.5 (numWorkers=8...

Reason sequential/test-worker-fshandles-error-on-termination
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51866/, https://github.com/nodejs/node/pull/52040/)
Appeared test-azure_msft-win2016_vs2017-x64-2, test-rackspace-win2022_vs2022-x64-6
First CI https://ci.nodejs.org/job/node-test-pull-request/57654/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57709/
Example
not ok 972 sequential/test-worker-fshandles-error-on-termination
  ---
  duration_ms: 874.23800
  severity: fail
  exitcode: 3221226356
  stack: |-
  ...


Reason wasi/test-wasi
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52010/, https://github.com/nodejs/node/pull/52028/)
Appeared test-azure_msft-win11_vs2022-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/57655/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57656/
Example
not ok 888 wasi/test-wasi
  ---
  duration_ms: 1140.02600
  severity: fail
  exitcode: 1
  stack: |-
    executing cant_dotdot
    (node:6112) ExperimentalWarning: WASI is an experimental feature and might change at any time
    (Use `node --trace-warnings ...` to show where the warning was created)
    fd_prestat_get(3, 69720)
    fd_prestat_dir_name(3, 131088, 8)
    fd_prestat_get(4, 69720)
    fd_prestat_dir_name(4, 131152, 4)
    fd_prestat_get(5, 69720)
    environ_sizes_get(69724, 69720)
    environ_get(138496, 131168)
    fd_fdstat_get(3, 69672)
    path_open(3, 1, 1033, 14, 0, 264240830, 268435455, 0, 69668)
    
    executing clock_getres
    (node:4712) ExperimentalWarning: WASI is an experimental feature and might change at any time
    (Use `node --trace-warnings ...` to show where the warning was created)
    fd_prestat_get(3, 69688)
    fd_prestat_dir_name(3, 131088, 8)
    fd_prestat_get(4, 69688)
    fd_prestat_dir_name(4, 131152, 4)
    fd_prestat_get(5, 69688)
    environ_sizes_get(69692, 6...

Git Failure
Build Failure
Reason ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/51866/, https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/52028/, https://github.com/nodejs/node/pull/52010/, https://github.com/nodejs/node/pull/52009/, https://github.com/nodejs/node/pull/51973/, https://github.com/nodejs/node/pull/51972/, https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/52027/, https://github.com/nodejs/node/pull/52037/)
Appeared test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win2016_vs2017-x64-1, test-azure_msft-win10_vs2019-x64-1, test-azure_msft-win11_vs2022-x64-4, test-azure_msft-win11_vs2022-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57657/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57712/
Example
ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52083/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57689/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57743/
Example
fatal: No rebase in progress?

Jenkins Failure
CCTest Failure
Progress
  • parallel/test-domain-error-types (12)
  • parallel/test-debugger-heap-profiler (10)
  • sequential/test-worker-fshandles-open-close-on-termination (3)
  • benchmark/test-benchmark-net (2)
  • parallel/test-blob-file-backed (2)
  • parallel/test-child-process-exec-encoding (2)
  • parallel/test-child-process-exec-maxbuf (2)
  • parallel/test-performance-function (2)
  • parallel/test-shadow-realm-gc-module (2)
  • parallel/test-tls-ticket-cluster (2)
  • parallel/test-worker-memory (2)
  • sequential/test-worker-fshandles-error-on-termination (2)
  • wasi/test-wasi (2)
  • ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error? (10)
  • fatal: No rebase in progress? (3)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked CI console logs and the named tests: parallel/test-domain-error-types, parallel/test-debugger-heap-profiler, sequential/test-worker-fshandles-open-close-on-termination, and benchmark/test-benchmark-net. Compare the repeated failures across the listed platforms and identify the underlying reliability issues. Done means the reported failures are addressed and the affected CI runs remain green.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.