nodejs / nodejs/reliability

CI Reliability 2023-01-21

Open
#490 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/49013 to node-test-pull-request/49111 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-01-21 00:13 1 4 28 2 65 4.12%
CCTest Failure
Reason [ RUN ] InspectorSocketTest.HostIPv6NonRoutable
Type CC_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/45796/, https://github.com/nodejs/node/pull/45770/, https://github.com/nodejs/node/pull/46237/, https://github.com/nodejs/node/pull/46227/)
Appeared test-azure_msft-win10_vs2019-x64-2, test-rackspace-win2012r2_vs2019-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/49032/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49110/
Example
[ RUN      ] InspectorSocketTest.HostIPv6NonRoutable
C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected)
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (5000 ms)

Reason ../test/cctest/test_inspector_socket.cc:323: Failure
Type CC_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46238/, https://github.com/nodejs/node/pull/46272/)
Appeared test-softlayer-alpine311_container-x64-1, test-digitalocean-alpine311_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49034/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49096/
Example
../test/cctest/test_inspector_socket.cc:323: Failure
Value of: (!expectation.read_expected)
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (4999 ms)

Jenkins Failure
Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/45942/)
Appeared test-softlayer-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49023/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49048/
Example
Build timed out (after 60 minutes). Marking the build as failed.

Git Failure
Build Failure
Reason Error: getWindowSize EINVAL
Type BUILD_FAILURE
Failed PR 13 (https://github.com/nodejs/node/pull/44943/, https://github.com/nodejs/node/pull/45258/, https://github.com/nodejs/node/pull/46251/, https://github.com/nodejs/node/pull/46250/, https://github.com/nodejs/node/pull/45772/, https://github.com/nodejs/node/pull/46253/, https://github.com/nodejs/node/pull/45936/, https://github.com/nodejs/node/pull/46031/, https://github.com/nodejs/node/pull/46272/, https://github.com/nodejs/node/pull/46199/, https://github.com/nodejs/node/pull/46174/, https://github.com/nodejs/node/pull/46269/, https://github.com/nodejs/node/pull/46286/)
Appeared test-equinix_mnx-smartos18-x64-4, test-equinix_mnx-smartos20-x64-4, test-equinix_mnx-smartos20-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/49013/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49107/
Example
Error: getWindowSize EINVAL
outline =     at WriteStream._refreshSize (tty.js:133:31)
outline =     at process.<anonymous> (internal/bootstrap/switches/is_main_thread.js:127:41)
outline =     at process.emit (events.js:400:28)
outline = Emitted 'error' event on WriteStream instance at:
outline =     at WriteStream._refreshSize (tty.js:133:10)

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 11 (https://github.com/nodejs/node/pull/45258/, https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46238/, https://github.com/nodejs/node/pull/45770/, https://github.com/nodejs/node/pull/45735/, https://github.com/nodejs/node/pull/46253/, https://github.com/nodejs/node/pull/46267/, https://github.com/nodejs/node/pull/45772/, https://github.com/nodejs/node/pull/45038/, https://github.com/nodejs/node/pull/46031/, https://github.com/nodejs/node/pull/46271/)
Appeared test-rackspace-fedora32-x64-1, test-osuosl-rhel8-ppc64_le-2, test-rackspace-centos7-x64-1, test-orka-macos10.15-x64-2, test-orka-macos11-x64-2, test-nearform-macos11.0-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49019/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49088/
Example
fatal: No rebase in progress?

JSTest Failure
Reason pummel/test-fs-readfile-tostring-fail
Type JS_TEST_FAILURE
Failed PR 9 (https://github.com/nodejs/node/pull/46174/, https://github.com/nodejs/node/pull/46213/, https://github.com/nodejs/node/pull/45735/, https://github.com/nodejs/node/pull/45038/, https://github.com/nodejs/node/pull/46237/, https://github.com/nodejs/node/pull/46267/, https://github.com/nodejs/node/pull/46271/, https://github.com/nodejs/node/pull/46276/, https://github.com/nodejs/node/pull/44943/)
Appeared test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49063/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49095/
Example
not ok 3477 pummel/test-fs-readfile-tostring-fail
  ---
  duration_ms: 0.196
  severity: fail
  exitcode: 7
  stack: |-
    /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/pummel/test-fs-readfile-tostring-fail.js:73
      throw err;
      ^
    
    AssertionError [ERR_ASSERTION]: ifError got unwanted exception: ENOSPC: no space left on device, write
        at Object.callback (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/pummel/test-fs-readfile-tostring-fail.js:31:37)
        at callback (node:internal/streams/writable:555:21)
        at onwriteError (node:internal/streams/writable:418:3)
        at onwrite (node:internal/streams/writable:460:7)
        at node:internal/fs/streams:439:14
        at FSReqCallback.wrapper [as oncomplete] (node:fs:938:5) {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      actual: [Error: ENOSPC: no space left on device, write] {
        errno: -28,
        code: 'ENOSPC',
        syscall: 'write'
      },
      expe...

Reason parallel/test-child-process-cwd
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/45258/, https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46258/, https://github.com/nodejs/node/pull/45966/, https://github.com/nodejs/node/pull/46249/, https://github.com/nodejs/node/pull/46263/)
Appeared test-nearform_arm-win10_vs2019-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49019/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49109/
Example
not ok 98 parallel/test-child-process-cwd
  ---
  duration_ms: 0.341
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    + actual - expected
    
    + 'C:\\Windows'
    - 'C:\\WINDOWS'
            ^
        at ChildProcess.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-cwd.js:56:26)
        at ChildProcess.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:448:15)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1098:16)
        at Socket.<anonymous> (node:internal/child_process:456:11)
        at Socket.emit (node:events:519:28)
        at Pipe.<anonymous> (node:net:315:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 'C:\\Windows',
      expected: 'C:\\WINDOWS',
      operator: 'strictEqual'
    }
    
...

Reason parallel/test-child-process-exec-cwd
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/45258/, https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46258/, https://github.com/nodejs/node/pull/45966/, https://github.com/nodejs/node/pull/46249/, https://github.com/nodejs/node/pull/46263/)
Appeared test-nearform_arm-win10_vs2019-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/49019/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49109/
Example
not ok 100 parallel/test-child-process-exec-cwd
  ---
  duration_ms: 0.350
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:400
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(stdout.startsWith(dir))
    
        at c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-child-process-exec-cwd.js:38:3
        at c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:413:17
        at c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:448:15
        at ChildProcess.exithandler (node:child_process:411:7)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1098:16)
        at ChildProcess._handle.onexit (node:internal/child_process:304:5) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v20.0.0-pre
  ...


Reason sequential/test-worker-prof
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/45258/, https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46258/, https://github.com/nodejs/node/pull/46174/, https://github.com/nodejs/node/pull/46249/, https://github.com/nodejs/node/pull/46263/)
Appeared test-nearform-macos11.0-arm64-1, test-azure_msft-win10_vs2019-x64-4, test-rackspace-win2012r2_vs2017-x64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-azure_msft-win10_vs2019-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49019/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49109/
Example
not ok 3882 sequential/test-worker-prof
  ---
  duration_ms: 0.520
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:400
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: 13 >= 15
        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/sequential/test-worker-prof.js:78:5)
        at Module._compile (node:internal/modules/cjs/loader:1246:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1300:10)
        at Module.load (node:internal/modules/cjs/loader:1103:32)
        at Module._load (node:internal/modules/cjs/loader:942:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
        at node:internal/main/run_main_module:23:47 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v20.0.0-pre
  ...


Reason parallel/test-performance-function
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46228/, https://github.com/nodejs/node/pull/45942/, https://github.com/nodejs/node/pull/46253/, https://github.com/nodejs/node/pull/45772/)
Appeared test-ibm-ubuntu1804-x64-2, undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/49023/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49067/
Example
not ok 2230 parallel/test-performance-function
  ---
  duration_ms: 0.172
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/errors:490
        ErrorCaptureStackTrace(err);
        ^
    
    RangeError [ERR_OUT_OF_RANGE]: The value of "val" is out of range. It must be >= 1 && <= 9007199254740991. Received 0
        at RecordableHistogram.record (node:internal/histogram:283:5)
        at processComplete (node:internal/perf/timerify:40:15)
        at timerified m (node:internal/perf/timerify:87:5)
        at /home/iojs/build/workspace/node-test-commit-linux/test/parallel/test-performance-function.js:98:5
        at runNextTicks (node:internal/process/task_queues:60:5)
        at listOnTimeout (node:internal/timers:537:9)
        at process.processTimers (node:internal/timers:511:7) {
      code: 'ERR_OUT_OF_RANGE'
    }
    
    Node.js v20.0.0-pre
  ...


Reason parallel/test-heapsnapshot-near-heap-limit-by-api-in-worker
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46271/, https://github.com/nodejs/node/pull/46272/)
Appeared test-orka-macos11-x64-2, test-equinix_mnx-smartos20-x64-3, undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/49020/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49096/
Example
not ok 944 parallel/test-heapsnapshot-near-heap-limit-by-api-in-worker
  ---
  duration_ms: 2.11
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:400
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert(stderr.includes('ERR_WORKER_OUT_OF_MEMORY'))
    
        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-heapsnapshot-near-heap-limit-by-api-in-worker.js:36:5)
        at Module._compile (node:internal/modules/cjs/loader:1238:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1292:10)
        at Module.load (node:internal/modules/cjs/loader:1095:32)
        at Module._load (node:internal/modules/cjs/loader:934:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
        at node:internal/main/run_main_module:23:47 {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      ...

Reason pummel/test-net-throttle
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/46238/, https://github.com/nodejs/node/pull/46242/, https://github.com/nodejs/node/pull/46271/)
Appeared test-digitalocean-freebsd12-x64-1, test-digitalocean-freebsd12-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/49015/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49088/
Example
not ok 3500 pummel/test-net-throttle
  ---
  duration_ms: 0.579
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
        at process.emit (node:events:530:35) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.0.0-pre
  ...


Reason sequential/test-child-process-pass-fd
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/46237/, https://github.com/nodejs/node/pull/46271/, https://github.com/nodejs/node/pull/46249/)
Appeared test-macstadium-macos11.0-arm64-3, test-macstadium-macos11.0-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/49074/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49097/
Example
not ok 3726 sequential/test-child-process-pass-fd
  ---
  duration_ms: 0.211
  severity: crashed
  exitcode: -10
  stack: |-
    node:events:496
          throw er; // Unhandled 'error' event
          ^
    
    Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
        at new NodeError (node:internal/errors:399:5)
        at target.send (node:internal/child_process:747:16)
        at Socket.socketConnected (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/sequential/test-child-process-pass-fd.js:62:15)
        at Object.onceWrapper (node:events:632:28)
        at Socket.emit (node:events:518:28)
        at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1479:10)
    Emitted 'error' event on process instance at:
        at node:internal/child_process:751:35
        at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
      code: 'ERR_IPC_CHANNEL_CLOSED'
    }
    
    Node.js v20.0.0-pre
    node:events:496
          throw er; // Unhandled 'error' event
    ...

Reason parallel/test-child-process-can-write-to-stdout
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44432/, https://github.com/nodejs/node/pull/46267/)
Appeared test-macstadium-macos11.0-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/49055/
Last CI https://ci.nodejs.org/job/node-test-pull-request/49103/
Example
not ok 134 parallel/test-child-process-can-write-to-stdout
  ---
  duration_ms: 0.93
  severity: crashed
  exitcode: -10
  stack: |-
  ...


Progress
  • [ RUN ] InspectorSocketTest.HostIPv6NonRoutable (4)
  • ../test/cctest/test_inspector_socket.cc:323: Failure (2)
  • Build timed out (after 60 minutes). Marking the build as failed. (2)
  • Error: getWindowSize EINVAL (13)
  • fatal: No rebase in progress? (11)
  • pummel/test-fs-readfile-tostring-fail (9)
  • parallel/test-child-process-cwd (6)
  • parallel/test-child-process-exec-cwd (6)
  • sequential/test-worker-prof (6)
  • parallel/test-performance-function (5)
  • parallel/test-heapsnapshot-near-heap-limit-by-api-in-worker (3)
  • pummel/test-net-throttle (3)
  • sequential/test-child-process-pass-fd (3)
  • parallel/test-child-process-can-write-to-stdout (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated reliability report and its linked CI jobs, then inspect the named failures in test/cctest/test_inspector_socket.cc, test/pummel/test-fs-readfile-tostring-fail.js, test/parallel/test-child-process-cwd.js, and test/parallel/test-child-process-exec-cwd.js. Done means the recurring failures in the report have a confirmed disposition and no longer reproduce in the relevant CI runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, javascript, node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.