nodejs / nodejs/reliability

CI Reliability 2024-03-16

Open
#813 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/57672 to node-test-pull-request/57768 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2024-03-16 00:08 1 0 32 6 61 0.00%
JSTest Failure
Reason parallel/test-domain-error-types
Type JS_TEST_FAILURE
Failed PR 8 (https://github.com/nodejs/node/pull/52036/, https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/51791/, https://github.com/nodejs/node/pull/52039/, https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52037/, https://github.com/nodejs/node/pull/52038/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-4, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-ubuntu2004_sharedlibs_container-arm64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/57676/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57718/
Example
not ok 945 parallel/test-domain-error-types
  ---
  duration_ms: 1441.34600
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-debugger-heap-profiler
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/52027/, https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/51866/, https://github.com/nodejs/node/pull/52023/, https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52040/)
Appeared test-azure_msft-win11_vs2022-arm64-4, test-azure_msft-win2016_vs2017-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57674/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57709/
Example
not ok 182 parallel/test-debugger-heap-profiler
  ---
  duration_ms: 2464.99000
  severity: fail
  exitcode: 1
  stack: |-
    Can't clean tmpdir: d:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.180
    Files blocking: [ 'node.heapsnapshot' ]
    
    d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:69
        throw e;
        ^
    
    Error: EBUSY: resource busy or locked, rmdir 'd:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.180'
        at rmdirSync (node:fs:1219:11)
        at _rmdirSync (node:internal/fs/rimraf:235:5)
        at rimrafSync (node:internal/fs/rimraf:193:7)
        at Object.rmSync (node:fs:1268:10)
        at rmSync (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:20:8)
        at onexit (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:54:5)
        at process.<anonymous> (d:\workspace\node-test-binary-windows-js-suites\node\test\common\tmpdir.js:43:14)
        at process.emit (...

Reason parallel/test-worker-memory
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/52077/, https://github.com/nodejs/node/pull/52071/, https://github.com/nodejs/node/pull/52101/)
Appeared test-azure_msft-win10_vs2019-x64-3, test-azure_msft-win2016_vs2017-x64-1, test-rackspace-win2022_vs2022-x64-6
First CI https://ci.nodejs.org/job/node-test-pull-request/57708/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57761/
Example
not ok 886 parallel/test-worker-memory
  ---
  duration_ms: 2110.14900
  severity: fail
  exitcode: 3221226356
  stack: |-
    run() called with n=15 (numWorkers=4)
    run() called with n=15 (numWorkers=4)
    run() called with n=15 (numWorkers=4)
    run() called with n=15 (numWorkers=4)
    run() called with n=14 (numWorkers=4)
    run() called with n=14 (numWorkers=4)
    run() called with n=14 (numWorkers=4)
    run() called with n=14 (numWorkers=4)
    run() called with n=13 (numWorkers=4)
    run() called with n=13 (numWorkers=4)
    run() called with n=13 (numWorkers=4)
    run() called with n=13 (numWorkers=4)
    run() called with n=12 (numWorkers=4)
    run() called with n=12 (numWorkers=4)
    run() called with n=12 (numWorkers=4)
    run() called with n=12 (numWorkers=4)
    run() called with n=11 (numWorkers=4)
    run() called with n=11 (numWorkers=4)
    run() called with n=11 (numWorkers=4)
    run() called with n=10 (numWorkers=4)
    run() called with n=10 (numWorkers=4)
    run() called wi...

Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52023/, https://github.com/nodejs/node/pull/51091/, https://github.com/nodejs/node/pull/51021/)
Appeared test-digitalocean-freebsd12-x64-2, test-azure_msft-win11_vs2022-x64-1, test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57688/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57768/
Example
not ok 2466 parallel/test-runner-output
  ---
  duration_ms: 26034.59100
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 14159.988486
          ...


Reason parallel/test-tls-ticket-cluster
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52016/, https://github.com/nodejs/node/pull/52077/, https://github.com/nodejs/node/pull/51136/)
Appeared test-nearform-macos10.15-x64-1, test-ibm-rhel8-x64-1, test-azure_msft-win11_vs2022-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57680/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57766/
Example
not ok 2808 parallel/test-tls-ticket-cluster
  ---
  duration_ms: 479.76500
  severity: fail
  exitcode: 1
  stack: |-
    [primary] got "listening"
    [primary] connecting 55824 session? false
    [primary] got "listening"
    [worker] connection reused? false
    [primary] got "not-reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 55824 session? true
    [worker] connection reused?...

Reason sequential/test-worker-fshandles-open-close-on-termination
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/52023/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52037/)
Appeared test-rackspace-win2019_vs2019-x64-3, test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win11_vs2022-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57688/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57713/
Example
not ok 973 sequential/test-worker-fshandles-open-close-on-termination
  ---
  duration_ms: 1050.04300
  severity: crashed
  exitcode: 3221225477
  stack: |-
  ...


Reason benchmark/test-benchmark-http2
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52037/, https://github.com/nodejs/node/pull/51136/)
Appeared test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57712/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57766/
Example
not ok 184 benchmark/test-benchmark-http2
  ---
  duration_ms: 1960.63900
  severity: fail
  exitcode: 1
  stack: |-
    Error: test-double-http2 failed with 3221226505.
        at ChildProcess.<anonymous> (C:\workspace\node-test-binary-windows-native-suites\node\benchmark\_http-benchmarkers.js:242:16)
        at Object.onceWrapper (node:events:632:26)
        at ChildProcess.emit (node:events:517:28)
        at maybeClose (node:internal/child_process:1098:16)
        at ChildProcess._handle.onexit (node:internal/child_process:303:5)
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    3221226505 !== 0
    
        at ChildProcess.<anonymous> (C:\workspace\node-test-binary-windows-native-suites\node\test\common\benchmark.js:28:12)
        at ChildProcess.emit (node:events:517:28)
        at ChildProcess._handle.onexit (node:internal/child_process:292:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTIO...

Reason parallel/test-fs-read-stream-concurrent-reads
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52077/, https://github.com/nodejs/node/pull/51091/)
Appeared test-equinix-ubuntu2004_container-armv7l-2, test-ibm-ubuntu2204_sharedlibs_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57741/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57751/
Example
not ok 1192 parallel/test-fs-read-stream-concurrent-reads
  ---
  duration_ms: 1229.63500
  severity: crashed
  exitcode: -11
  stack: |-
  ...


Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51050/, https://github.com/nodejs/node/pull/51977/)
Appeared test-ibm-rhel8-x64-1, test-ibm-rhel8-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57736/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57748/
Example
not ok 2346 parallel/test-performance-function
  ---
  duration_ms: 195.65100
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/histogram:291
        validateInteger(val, 'val', 1);
        ^
    
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:291:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-performance-function.js:103:5 {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v22.0.0-pre
  ...


Reason parallel/test-shadow-realm-gc-module
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52024/, https://github.com/nodejs/node/pull/51362/)
Appeared test-digitalocean-ubi81_container-x64-1, test-ibm-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57700/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57722/
Example
not ok 2431 parallel/test-shadow-realm-gc-module
  ---
  duration_ms: 5262.37100
  severity: crashed
  exitcode: -6
  stack: |-
    
    <--- Last few GCs --->
    ) [2298362:0x6515000]     4715 ms: Mark-Compact 19.1 (32.4) -> 16.5 (33.4) MB, pooled: 0 MB, 12.07 / 0.01 ms  (+ 16.1 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 43 ms) (average mu = 0.627, current mu = 0.661) [2298362:0x6515000]     4886 ms: Mark-Compact (reduce) 23.6 (34.4) -> 20.1 (21.9) MB, pooled: 0 MB, 38.68 / 0.01 ms  (+ 43.8 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 129 ms) (average mu = 0.620, current mu
    
    <--- JS stacktrace --->
    
    FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
    ----- Native stack trace -----
    
     1: 0xd70c70 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [out/Release/node]
     2: 0x10dc910 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OO...

Reason sequential/test-http-regr-gh-2928
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52038/, https://github.com/nodejs/node/pull/51136/)
Appeared test-digitalocean-freebsd12-x64-1, test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57716/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57766/
Example
not ok 3915 sequential/test-http-regr-gh-2928
  ---
  duration_ms: 120076.64400
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+....

Git Failure
Build Failure
Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/52040/, https://github.com/nodejs/node/pull/52051/, https://github.com/nodejs/node/pull/52083/, https://github.com/nodejs/node/pull/45427/, https://github.com/nodejs/node/pull/51362/, https://github.com/nodejs/node/pull/52101/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-digitalocean-alpine318_container-x64-1, test-digitalocean-debian12-x64-1, test-digitalocean-alpine318_container-x64-2, test-softlayer-debian12-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/57689/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57761/
Example
fatal: No rebase in progress?

Reason ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for ...
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45427/, https://github.com/nodejs/node/pull/51362/)
Appeared test-digitalocean-fedora37-x64-1, test-digitalocean-fedora38-x64-2, test-rackspace-fedora32-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57755/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57759/
Example
ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for node-test-commit-linux/nodes=fedora-last-latest-x64 #56651
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/52027/, https://github.com/nodejs/node/pull/52037/)
Appeared test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/57674/
Last CI https://ci.nodejs.org/job/node-test-pull-request/57712/
Example
ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Jenkins Failure
CCTest Failure
Progress
  • parallel/test-domain-error-types (8)
  • parallel/test-debugger-heap-profiler (7)
  • parallel/test-worker-memory (4)
  • parallel/test-runner-output (3)
  • parallel/test-tls-ticket-cluster (3)
  • sequential/test-worker-fshandles-open-close-on-termination (3)
  • benchmark/test-benchmark-http2 (2)
  • parallel/test-fs-read-stream-concurrent-reads (2)
  • parallel/test-performance-function (2)
  • parallel/test-shadow-realm-gc-module (2)
  • sequential/test-http-regr-gh-2928 (2)
  • fatal: No rebase in progress? (6)
  • ERROR: Step ‘Record fingerprints of files to track usage’ failed: no workspace for ... (2)
  • ERROR: Step �Publish JUnit test result report� failed: No test report files were found. Configuration error? (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked CI console logs for the listed failing tests, including parallel/test-domain-error-types and parallel/test-debugger-heap-profiler, and compare their failure patterns across platforms. Trace each test to its implementation and determine the relevant failure causes; done means the reported failures are understood and the affected CI tests are reliable again.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.