CI Reliability 2022-12-03

Open
#447 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
javascript, node.js, shell

Research direction

Start by reviewing the linked CI runs and the listed failing tests, including test-fs-watch-recursive, test-process-wrap, test-child-process-stdio-inherit, and test-watch-mode-files_watcher. Compare the first and last CI examples for recurring failures across platforms. Done means the reported CI failures have a concrete resolution and the affected tests no longer fail spuriously.

Written by the indexing model from the issue text.

Description

Failures in node-test-pull-request/48184 to node-test-pull-request/48280 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2022-12-03 00:12 6 5 30 5 54 5.62%
JSTest Failure
Reason parallel/test-fs-watch-recursive
Type JS_TEST_FAILURE
Failed PR 14 (https://github.com/nodejs/node/pull/45620/, https://github.com/nodejs/node/pull/45621/, https://github.com/nodejs/node/pull/45501/, https://github.com/nodejs/node/pull/45638/, https://github.com/nodejs/node/pull/44731/, https://github.com/nodejs/node/pull/45646/, https://github.com/nodejs/node/pull/45486/, https://github.com/nodejs/node/pull/45634/, https://github.com/nodejs/node/pull/45672/, https://github.com/nodejs/node/pull/45671/, https://github.com/nodejs/node/pull/45526/, https://github.com/nodejs/node/pull/45693/, https://github.com/nodejs/node/pull/43697/, https://github.com/nodejs/node/pull/45665/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-orka-macos11-x64-2, test-ibm-rhel8-s390x-4, test-digitalocean-alpine311_container-x64-2, test-digitalocean-ubi81_container-x64-2, test-softlayer-ubuntu1804_sharedlibs_container-x64-2, test-orka-macos11-x64-1, test-nearform-macos11.0-arm64-1, test-ibm-rhel8-s390x-1, test-azure_msft-win10_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/48195/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48263/
Example
not ok 1254 parallel/test-fs-watch-recursive
  ---
  duration_ms: 120.115
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    (node:19709) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added to [process]. Use emitter.setMaxListeners() to increase limit
    (Use `node --trace-warnings ...` to show where the warning was created)
  ...


Reason parallel/test-process-wrap
Type JS_TEST_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/45589/, https://github.com/nodejs/node/pull/45597/, https://github.com/nodejs/node/pull/45615/, https://github.com/nodejs/node/pull/45634/, https://github.com/nodejs/node/pull/45653/, https://github.com/nodejs/node/pull/45633/, https://github.com/nodejs/node/pull/45526/, https://github.com/nodejs/node/pull/45672/, https://github.com/nodejs/node/pull/45701/, https://github.com/nodejs/node/pull/45665/)
Appeared test-nearform-macos10.15-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/48184/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48279/
Example
not ok 2481 parallel/test-process-wrap
  ---
  duration_ms: 120.181
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    (node:42433) internal/test/binding: These APIs are for internal testing only. Do not use them.
    (Use `node --trace-warnings ...` to show where the warning was created)
  ...


Reason parallel/test-child-process-stdio-inherit
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/45590/, https://github.com/nodejs/node/pull/45615/, https://github.com/nodejs/node/pull/45501/, https://github.com/nodejs/node/pull/45500/, https://github.com/nodejs/node/pull/45526/, https://github.com/nodejs/node/pull/43697/, https://github.com/nodejs/node/pull/45586/)
Appeared test-nearform-macos10.15-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/48185/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48259/
Example
not ok 776 parallel/test-child-process-stdio-inherit
  ---
  duration_ms: 120.113
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason addons/report-fatalerror/test
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45634/, https://github.com/nodejs/node/pull/45676/)
Appeared test-joyent-smartos20-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/48212/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48233/
Example
not ok 3501 addons/report-fatalerror/test
  ---
  duration_ms: 300.157
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-child-process-exec-timeout-kill
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45621/, https://github.com/nodejs/node/pull/43697/)
Appeared test-nearform-macos10.15-x64-1, test-digitalocean-freebsd12-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/48197/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48257/
Example
not ok 160 parallel/test-child-process-exec-timeout-kill
  ---
  duration_ms: 0.468
  severity: fail
  exitcode: 1
  stack: |-
    [stdout] 
    [stderr] 
    node:assert:124
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    1 !== null
    
        at /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/parallel/test-child-process-exec-timeout-kill.js:32:10
        at /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/common/index.js:448:15
        at ChildProcess.exithandler (node:child_process:427:5)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1098:16)
        at Socket.<anonymous> (node:internal/child_process:456:11)
        at Socket.emit (node:events:519:28)
        at Pipe.<anonymous> (node:net:298:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: 1,
      expected: null,
      operator: 'strictEqual'
    ...

Reason parallel/test-fs-watch-recursive-symlink
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45638/, https://github.com/nodejs/node/pull/45693/)
Appeared test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/48216/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48254/
Example
not ok 1348 parallel/test-fs-watch-recursive-symlink
  ---
  duration_ms: 120.79
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-runner-inspect
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45671/, https://github.com/nodejs/node/pull/45665/)
Appeared test-nearform-macos10.15-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/48247/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48263/
Example
not ok 2662 parallel/test-runner-inspect
  ---
  duration_ms: 120.66
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] (node:50255) Warning: Using the inspector with --test forces running at a concurrency of 1. Use the inspectPort option to run with concurrency
    [err] (Use `node --trace-warnings ...` to show where the warning was created)
    [err] 
    [out] TAP version 13
    [out] 
    [err] Debugger listening on ws://127.0.0.1:50899/2678ba3b-c74a-468d-a90e-afd28d72676a
    [err] 
    [err] Debugger listening on ws://127.0.0.1:50899/2678ba3b-c74a-468d-a90e-afd28d72676a
    [err] 
    [out] # Subtest: /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/test-runner/index.test.js
    [out] 
    [out]     # Debugger listening on ws://127.0.0.1:50899/2678ba3b-c74a-468d-a90e-afd28d72676a
    [out] 
    [out]     # For help, see: https://nodejs.org/en/docs/inspector
    [out] 
    [err] Debugge...

Reason parallel/test-watch-mode-files_watcher
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45652/, https://github.com/nodejs/node/pull/45665/)
Appeared test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/48217/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48280/
Example
not ok 3431 parallel/test-watch-mode-files_watcher
  ---
  duration_ms: 120.30
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    TAP version 13
    # Subtest: watch mode file watcher
        # Subtest: should watch changed files
        ok 1 - should watch changed files
          ---
          duration_ms: 202.958812
          ...


Reason sequential/test-watch-mode
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44873/, https://github.com/nodejs/node/pull/45660/)
Appeared test-requireio_securogroup-debian10-arm64_pi3-1, test-requireio_joeyvandijk-debian10-armv7l_pi2-2
First CI https://ci.nodejs.org/job/node-test-pull-request/48214/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48226/
Example
not ok 886 sequential/test-watch-mode
  ---
  duration_ms: 13.500
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: watch mode
        # Subtest: should watch changes to a file - event loop ended
        ok 1 - should watch changes to a file - event loop ended
          ---
          duration_ms: 7.259099031
          ...


Jenkins Failure
Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/45633/, https://github.com/nodejs/node/pull/45672/)
Appeared test-softlayer-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/48229/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48246/
Example
Build timed out (after 60 minutes). Marking the build as failed.

Git Failure
Build Failure
Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/45615/, https://github.com/nodejs/node/pull/45653/, https://github.com/nodejs/node/pull/45597/)
Appeared test-rackspace-win2012r2_vs2017-x64-1, test-rackspace-win2012r2_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/48190/
Last CI https://ci.nodejs.org/job/node-test-pull-request/48220/
Example
error: remote GitHub already exists.

> git fetch --prune --no-tags GitHub 

> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp

CCTest Failure
Progress
  • parallel/test-fs-watch-recursive (14)
  • parallel/test-process-wrap (10)
  • parallel/test-child-process-stdio-inherit (7)
  • addons/report-fatalerror/test (2)
  • parallel/test-child-process-exec-timeout-kill (2)
  • parallel/test-fs-watch-recursive-symlink (2)
  • parallel/test-runner-inspect (2)
  • parallel/test-watch-mode-files_watcher (2)
  • sequential/test-watch-mode (2)
  • Build timed out (after 60 minutes). Marking the build as failed. (2)
  • error: remote GitHub already exists. (3)
Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from nodejs/reliability

All issues in nodejs/reliability

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.