nodejs / nodejs/reliability

CI Reliability 2023-05-15

Open
#566 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/51700 to node-test-pull-request/51799 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-05-15 00:14 0 7 34 1 58 7.07%
CCTest Failure
Reason ../test/cctest/test_inspector_socket.cc:323: Failure
Type CC_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47943/, https://github.com/nodejs/node/pull/47991/)
Appeared test-softlayer-alpine311_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51727/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51780/
Example
../test/cctest/test_inspector_socket.cc:323: Failure
Value of: (!expectation.read_expected)
  Actual: true
Expected: false
[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (4999 ms)

Jenkins Failure
Git Failure
Build Failure
Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47939/, https://github.com/nodejs/node/pull/47991/, https://github.com/nodejs/node/pull/47997/)
Appeared test-equinix_mnx-smartos20-x64-3, test-osuosl-aix72-ppc64_be-3, test-equinix-rhel8_container-arm64-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-equinix-ubuntu2004_container-arm64-2, test-osuosl-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-osuosl-aix72-ppc64_be-2, test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51759/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51797/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47927/, https://github.com/nodejs/node/pull/45629/, https://github.com/nodejs/node/pull/47997/)
Appeared test-rackspace-win2012r2_vs2019-x64-6, test-rackspace-win2012r2_vs2019-x64-2, test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/51722/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51797/
Example
error: remote GitHub already exists.

> git fetch --prune --no-tags GitHub 

> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp

Reason sh: line 12: [: -gt: unary operator expected
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47860/, https://github.com/nodejs/node/pull/45629/, https://github.com/nodejs/node/pull/47997/)
Appeared test-rackspace-win2012r2_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51753/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51797/
Example
sh: line 12: [: -gt: unary operator expected

Reason ERROR: No files found with the specified search criteria.
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47860/, https://github.com/nodejs/node/pull/47997/)
Appeared test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/51753/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51797/
Example
ERROR: No files found with the specified search criteria.
        1 file(s) copied.

> set DEBUG_HELPER=1 

> call vcbuild.bat arm64 release msi

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47927/, https://github.com/nodejs/node/pull/47997/)
Appeared test-nearform-macos10.15-x64-1, test-orka-macos11-x64-1, test-softlayer-alpine311_container-x64-1, test-digitalocean-alpine312_container-x64-2, test-rackspace-debian10-x64-1, test-digitalocean-fedora32-x64-1, test-digitalocean-fedora34-x64-1, test-ibm-rhel8-x64-1, test-softlayer-ubi81_container-x64-1, test-osuosl-rhel8-ppc64_le-2, test-nearform-macos11.0-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51722/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51797/
Example
fatal: No rebase in progress?

JSTest Failure
Reason es-module/test-esm-loader-http-imports
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/47727/, https://github.com/nodejs/node/pull/47927/, https://github.com/nodejs/node/pull/47946/, https://github.com/nodejs/node/pull/47957/, https://github.com/nodejs/node/pull/47983/, https://github.com/nodejs/node/pull/45629/)
Appeared test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51717/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51778/
Example
not ok 3348 es-module/test-esm-loader-http-imports
  ---
  duration_ms: 26494.36100
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: ESM: http import via loader
        # Subtest: should load using --import flag
        not ok 1 - should load using --import flag
          ---
          duration_ms: 24569.780565
          failureType: 'testCodeFailure'
          error: |-
            Expected values to be strictly equal:
            + actual - expected
            
            + '\n' +
            +   'node:internal/process/esm_loader:42\n' +
            +   "      internalBinding('errors').triggerUncaughtException(\n" +
            +   '                                ^\n' +
            +   'Error: read ECONNRESET\n' +
            +   '    at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {\n' +
            +   '  errno: -54,\n' +
            +   "  code: 'ECONNRESET',\n" +
            +   "  syscall: 'read'\n" +
            +   '}\n' +
            +   '\n' +
         ...

Reason pummel/test-net-throttle
Type JS_TEST_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/47933/, https://github.com/nodejs/node/pull/47909/, https://github.com/nodejs/node/pull/47860/, https://github.com/nodejs/node/pull/47983/, https://github.com/nodejs/node/pull/47949/, https://github.com/nodejs/node/pull/47885/)
Appeared test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51711/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51781/
Example
not ok 3588 pummel/test-net-throttle
  ---
  duration_ms: 541.63700
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at process.<anonymous> (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd12-x64/test/pummel/test-net-throttle.js:87:10)
        at process.emit (node:events:523:35) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v21.0.0-pre
  ...


Reason parallel/test-http-remove-connection-header-persists-connection
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/47919/, https://github.com/nodejs/node/pull/47931/, https://github.com/nodejs/node/pull/47364/, https://github.com/nodejs/node/pull/47991/, https://github.com/nodejs/node/pull/47994/)
Appeared test-orka-macos11-x64-1, test-digitalocean-ubi81_container-x64-1, test-orka-macos11-x64-2, test-digitalocean-ubi81_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/51701/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51783/
Example
not ok 1234 parallel/test-http-remove-connection-header-persists-connection
  ---
  duration_ms: 597.22600
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:67:16
        at Timeout._onTimeout (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-http-remove-connection-header-persists-connection.js:54:7)
        at listOnTimeout (node:internal/timers:573:17)
        at process.processTimers (node:internal/timers:514:7) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v21.0.0-pre
  ...


Reason parallel/test-http2-misbehaving-multiplex
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47933/, https://github.com/nodejs/node/pull/47943/, https://github.com/nodejs/node/pull/47977/)
Appeared test-orka-macos11-x64-2, test-ibm-rhel8-s390x-4, test-orka-macos11-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51715/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51775/
Example
not ok 1481 parallel/test-http2-misbehaving-multiplex
  ---
  duration_ms: 537.18400
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected "actual" to be strictly unequal to: 5
        at Http2Server.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/parallel/test-http2-misbehaving-multiplex.js:45:10)
        at Http2Server.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx11-x64/test/common/index.js:465:15)
        at Http2Server.emit (node:events:511:28)
        at ServerHttp2Session.sessionOnStream (node:internal/http2/core:3003:19)
        at ServerHttp2Session.emit (node:events:511:28)
        at emit (node:internal/http2/core:331:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:85:22) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 5,
      expected: 5,
      operator: 'notStrictEqual'
   ...

Reason parallel/test-runner-watch-mode
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47881/, https://github.com/nodejs/node/pull/47939/, https://github.com/nodejs/node/pull/47998/)
Appeared test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-6, test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51706/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51782/
Example
not ok 3550 parallel/test-runner-watch-mode
  ---
  duration_ms: 960070.21300
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    TAP version 13
    # Subtest: test runner watch mode
        # Subtest: should run tests repeatedly
        ok 1 - should run tests repeatedly
          ---
          duration_ms: 719.31042
          ...


Reason parallel/test-x509-escaping
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/47933/, https://github.com/nodejs/node/pull/47931/)
Appeared test-rackspace-win2012r2_vs2019-x64-6, test-rackspace-win2012r2_vs2015-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/51719/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51744/
Example
not ok 885 parallel/test-x509-escaping
  ---
  duration_ms: 5805.00300
  severity: fail
  exitcode: 1
  stack: |-
    node:events:489
          throw er; // Unhandled 'error' event
          ^
    
    Error [ERR_SOCKET_CONNECTION_TIMEOUT]: Socket connection timeout
        at new NodeError (node:internal/errors:399:5)
        at internalConnectMultiple (node:net:1099:20)
        at Timeout.internalConnectMultipleTimeout (node:net:1638:3)
        at listOnTimeout (node:internal/timers:575:11)
        at process.processTimers (node:internal/timers:514:7)
    Emitted 'error' event on TLSSocket instance at:
        at emitErrorNT (node:internal/streams/destroy:151:8)
        at emitErrorCloseNT (node:internal/streams/destroy:116:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      code: 'ERR_SOCKET_CONNECTION_TIMEOUT'
    }
    
    Node.js v21.0.0-pre
  ...


Reason sequential/test-net-connect-econnrefused
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/47867/, https://github.com/nodejs/node/pull/47997/, https://github.com/nodejs/node/pull/48009/)
Appeared test-ibm-rhel7-s390x-3
First CI https://ci.nodejs.org/job/node-test-pull-request/51793/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51798/
Example
not ok 3710 sequential/test-net-connect-econnrefused
  ---
  duration_ms: 120.101
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    pending 50 rounds 1
    pending 49 rounds 1
    pending 48 rounds 1
    pending 47 rounds 1
    pending 46 rounds 1
    pending 45 rounds 1
    pending 44 rounds 1
    pending 43 rounds 1
    pending 42 rounds 1
    pending 41 rounds 1
    pending 40 rounds 1
    pending 39 rounds 1
    pending 38 rounds 1
    pending 37 rounds 1
    pending 36 rounds 1
    pending 35 rounds 1
    pending 34 rounds 1
    pending 33 rounds 1
    pending 32 rounds 1
    pending 31 rounds 1
    pending 30 rounds 1
    pending 29 rounds 1
    pending 28 rounds 1
    pending 27 rounds 1
    pending 26 rounds 1
    pending 25 rounds 1
    pending 24 rounds 1
    pending 23 rounds 1
    pending 22 rounds 1
    pending 21 rounds 1
    pending 20 rounds 1
    pending 19 rounds 1
    pending 18 rounds 1
    pending 17 rounds 1
    pending 16 rounds 1
    pending 15 rounds 1
    pending 14 rounds...

Reason parallel/test-fs-watch-recursive-symlink
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47650/, https://github.com/nodejs/node/pull/47939/)
Appeared test-ibm-rhel8-s390x-3, test-ibm-rhel8-s390x-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51720/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51730/
Example
not ok 1343 parallel/test-fs-watch-recursive-symlink
  ---
  duration_ms: 120040.27600
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-inspector-async-stack-traces-promise-then
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47931/, https://github.com/nodejs/node/pull/47991/)
Appeared test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/51732/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51780/
Example
not ok 2004 parallel/test-inspector-async-stack-traces-promise-then
  ---
  duration_ms: 15296.71900
  severity: fail
  exitcode: 1
  stack: |-
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:45243/934f9a67-aaf4-42ee-9f29-50ac6a9c003b
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
    Timed out waiting for matching notification (break on [eval]:0)
    1
  ...


Reason parallel/test-inspector-debug-end
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47055/, https://github.com/nodejs/node/pull/45629/)
Appeared test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/51719/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51770/
Example
not ok 2771 parallel/test-inspector-debug-end
  ---
  duration_ms: 120073.39400
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    Test there's no crash stopping server that was not started
    Test there's no crash stopping server without connecting
    [err] Debugger listening on ws://127.0.0.1:35517/b91d9414-9655-4044-853f-2b2e90c916c1
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    Test there's no crash stopping server after connecting
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:44563/01abed30-af5f-4de0-81c7-d5ba92958139
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
  ...


Reason sequential/test-http-regr-gh-2928
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/47364/, https://github.com/nodejs/node/pull/47885/)
Appeared test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/51746/
Last CI https://ci.nodejs.org/job/node-test-pull-request/51781/
Example
not ok 3894 sequential/test-http-regr-gh-2928
  ---
  duration_ms: 120075.42300
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+....

Progress
  • ../test/cctest/test_inspector_socket.cc:323: Failure (2)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (3)
  • error: remote GitHub already exists. (3)
  • sh: line 12: [: -gt: unary operator expected (3)
  • ERROR: No files found with the specified search criteria. (2)
  • fatal: No rebase in progress? (2)
  • es-module/test-esm-loader-http-imports (6)
  • pummel/test-net-throttle (6)
  • parallel/test-http-remove-connection-header-persists-connection (5)
  • parallel/test-http2-misbehaving-multiplex (3)
  • parallel/test-runner-watch-mode (3)
  • parallel/test-x509-escaping (3)
  • sequential/test-net-connect-econnrefused (3)
  • parallel/test-fs-watch-recursive-symlink (2)
  • parallel/test-inspector-async-stack-traces-promise-then (2)
  • parallel/test-inspector-debug-end (2)
  • sequential/test-http-regr-gh-2928 (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report is generated by ncu-ci walk pr --stats=true and links Jenkins CI jobs, including node-test-pull-request. Start by grouping the listed CCTest, build, and JavaScript test failures and inspecting their example logs, such as test/cctest/test_inspector_socket.cc and test/pummel/test-net-throttle.js. The issue does not define one requested change; done would require a scoped failure fix validated by the affected CI jobs.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github, node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.