nodejs / nodejs/reliability

CI Reliability 2023-09-13

Open
#664 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/53822 to node-test-pull-request/53918 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-09-13 00:12 1 3 41 2 53 3.09%
JSTest Failure
Reason parallel/test-runner-watch-mode
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/49565/, https://github.com/nodejs/node/pull/49575/, https://github.com/nodejs/node/pull/49582/, https://github.com/nodejs/node/pull/48510/, https://github.com/nodejs/node/pull/49578/)
Appeared test-azure_msft-win11_vs2022-arm64-5, test-osuosl-ubuntu2004_container-armv7l-1, test-rackspace-win2012r2_vs2015-x64-1, test-rackspace-win2022_vs2022-x64-5, test-azure_msft-win2016_vs2017-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53822/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53918/
Example
not ok 679 parallel/test-runner-watch-mode
  ---
  duration_ms: 4692.99100
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner watch mode
        # Subtest: should run tests repeatedly
        ok 1 - should run tests repeatedly
          ---
          duration_ms: 1597.4058
          ...


Reason parallel/test-tls-socket-close
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/49567/, https://github.com/nodejs/node/pull/49532/, https://github.com/nodejs/node/pull/49144/, https://github.com/nodejs/node/pull/49543/, https://github.com/nodejs/node/pull/49620/)
Appeared test-nearform-macos11.0-arm64-1, test-ibm-rhel8-s390x-2, test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/53843/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53912/
Example
not ok 2848 parallel/test-tls-socket-close
  ---
  duration_ms: 135.04000
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    false !== true
    
        at Immediate._onImmediate (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/parallel/test-tls-socket-close.js:47:16)
        at process.processImmediate (node:internal/timers:478:21) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: 'strictEqual'
    }
    
    Node.js v20.6.2-pre
  ...


Reason sequential/test-single-executable-application-snapshot-and-code-cache
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/49419/, https://github.com/nodejs/node/pull/49582/, https://github.com/nodejs/node/pull/49574/, https://github.com/nodejs/node/pull/49621/, https://github.com/nodejs/node/pull/49627/)
Appeared test-azure_msft-win2016_vs2017-x64-3, test-rackspace-debian10-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53835/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53914/
Example
not ok 943 sequential/test-single-executable-application-snapshot-and-code-cache
  ---
  duration_ms: 19060.87800
  severity: fail
  exitcode: 1
  stack: |-
    [process 1904]: --- stderr ---
    Aborted()
    
    [process 1904]: --- stdout ---
    Start injection of NODE_SEA_BLOB in C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.942\sea.exe...
    Error: Aborted(). Build with -sASSERTIONS for more info.
    
    [process 1904]: status = 1, signal = null
    C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86
        throw new Error(`${failures.join('\n')}`);
        ^
    
    Error: - process terminated with status 1, expected 0
        at logAndThrow (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:86:11)
        at expectSyncExit (C:\workspace\node-test-binary-windows-js-suites\node\test\common\child_process.js:91:5)
        at spawnSyncAndExitWithoutError (C:\workspace\node-test-binary-windows-js-suites\...

Reason sequential/test-tls-psk-client
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/49574/, https://github.com/nodejs/node/pull/49587/, https://github.com/nodejs/node/pull/49611/)
Appeared test-ibm-aix72-ppc64_be-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53839/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53895/
Example
not ok 3888 sequential/test-tls-psk-client
  ---
  duration_ms: 5600.73800
  severity: fail
  exitcode: 1
  stack: |-
    Failed: Timed out
    Failed: Error: connect ECONNREFUSED 127.0.0.1:12346
        at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
      errno: -79,
      code: 'ECONNREFUSED',
      syscall: 'connect',
      address: '127.0.0.1',
      port: 12346
    }
  ...


Reason parallel/test-crypto-dh-generate-keys
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49542/, https://github.com/nodejs/node/pull/49237/)
Appeared test-rackspace-win2012r2_vs2015-x64-2, test-azure_msft-win2016_vs2017-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/53825/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53889/
Example
not ok 916 parallel/test-crypto-dh-generate-keys
  ---
  duration_ms: 120141.99800
  severity: fail
  stack: |-
    timeout
  ...


Reason parallel/test-debugger-break
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49532/, https://github.com/nodejs/node/pull/49589/)
Appeared test-rackspace-win2022_vs2022-x64-5, test-rackspace-win2012r2_vs2019-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53851/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53875/
Example
not ok 165 parallel/test-debugger-break
  ---
  duration_ms: 816.03400
  severity: fail
  exitcode: 1
  stack: |-
    node:internal/process/promises:288
                triggerUncaughtException(err, true /* fromPromise */);
                ^
    
    AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
    
      assert.ok(cli.output.includes(`#0 ${script}:6`))
    
        at C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-debugger-break.js:63:10
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v18.17.2-pre
  ...


Reason parallel/test-inspector-debug-end
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49575/, https://github.com/nodejs/node/pull/49620/)
Appeared test-osuosl-rhel8-ppc64_le-2, test-ibm-rhel8-s390x-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53842/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53912/
Example
not ok 2944 parallel/test-inspector-debug-end
  ---
  duration_ms: 120088.62600
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    Test there's no crash stopping server that was not started
    Test there's no crash stopping server without connecting
    [err] Debugger listening on ws://127.0.0.1:37909/9e6fda0c-c60f-454f-a11e-a5f2c5f1b9f5
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    Test there's no crash stopping server after connecting
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:36641/e189ff87-b900-4dcd-8e80-64a3af1f557c
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
  ...


Reason parallel/test-runner-output
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49563/, https://github.com/nodejs/node/pull/49618/)
Appeared test-osuosl-aix72-ppc64_be-3, undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/53845/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53907/
Example
not ok 2483 parallel/test-runner-output
  ---
  duration_ms: 3966.68200
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: test runner output
        # Subtest: test-runner/output/abort.js
        ok 1 - test-runner/output/abort.js
          ---
          duration_ms: 2155.196812
          ...


Reason parallel/test-tls-ticket-cluster
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49558/, https://github.com/nodejs/node/pull/49581/)
Appeared test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53832/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53850/
Example
not ok 777 parallel/test-tls-ticket-cluster
  ---
  duration_ms: 775.00000
  severity: fail
  exitcode: 1
  stack: |-
    [primary] got "listening"
    [primary] connecting 64762 session? false
    [primary] got "listening"
    [primary] got "listening"
    [primary] got "listening"
    [worker] connection reused? false
    [primary] got "not-reused"
    [primary] connecting 64762 session? true
    [primary] connecting 64762 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 64762 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 64762 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 64762 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 64762 session? true
    [worker] connection reused? true
    [primary] got "reused"
    [primary] connecting 64762 session? true
    [worker] connection reused? true...

Reason sequential/test-cli-syntax-require
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49568/, https://github.com/nodejs/node/pull/49569/)
Appeared test-ibm-rhel8-x64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/53827/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53830/
Example
not ok 3916 sequential/test-cli-syntax-require
  ---
  duration_ms: 183.53400
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:1026
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^SyntaxError: \b/m. Input:
    
    ''
    
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/sequential/test-cli-syntax-require.js:34:14)
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux/test/common/index.js:474:15)
        at ChildProcess.emit (node:events:514:28)
        at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: '',
      expected: /^SyntaxError: \b/m,
      operator: 'match'
    }
    
    Node.js v21.0.0-pre
  ...


Reason sequential/test-http-regr-gh-2928
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49570/, https://github.com/nodejs/node/pull/49543/)
Appeared test-rackspace-win2012r2_vs2015-x64-1, test-rackspace-win2012r2_vs2015-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53834/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53887/
Example
not ok 934 sequential/test-http-regr-gh-2928
  ---
  duration_ms: 639.98900
  severity: fail
  exitcode: 3221226505
  stack: |-
    .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
  ...


Jenkins Failure
CCTest Failure
Git Failure
Build Failure
Reason Failed to trigger sub builds
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49255/, https://github.com/nodejs/node/pull/48638/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/53857/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53897/
Example
Failed to trigger sub builds

Reason error: The last gc run reported the following. Please correct the root cause
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49568/, https://github.com/nodejs/node/pull/49025/)
Appeared test-ibm-ubuntu1804-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10
First CI https://ci.nodejs.org/job/node-test-pull-request/53873/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53880/
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Reason fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49567/, https://github.com/nodejs/node/pull/48510/)
Appeared test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53843/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53915/
Example
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.

Progress
  • parallel/test-runner-watch-mode (5)
  • parallel/test-tls-socket-close (5)
  • sequential/test-single-executable-application-snapshot-and-code-cache (5)
  • sequential/test-tls-psk-client (3)
  • parallel/test-crypto-dh-generate-keys (2)
  • parallel/test-debugger-break (2)
  • parallel/test-inspector-debug-end (2)
  • parallel/test-runner-output (2)
  • parallel/test-tls-ticket-cluster (2)
  • sequential/test-cli-syntax-require (2)
  • sequential/test-http-regr-gh-2928 (2)
  • Failed to trigger sub builds (2)
  • error: The last gc run reported the following. Please correct the root cause (2)
  • fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the listed failing tests, including parallel/test-runner-watch-mode, parallel/test-tls-socket-close, sequential/test-tls-psk-client, and parallel/test-crypto-dh-generate-keys, then inspect their example CI logs and reproduce the failures on the affected platforms. Done means the reported intermittent failures have an identified cause and the affected tests no longer fail across the relevant CI runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.