nodejs / nodejs/reliability

CI Reliability 2023-12-25

Open
#750 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/56418 to node-test-pull-request/56517 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-12-25 00:13 0 13 18 11 58 14.61%
JSTest Failure
Reason parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/51244/, https://github.com/nodejs/node/pull/51138/, https://github.com/nodejs/node/pull/51255/, https://github.com/nodejs/node/pull/51257/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-2, test-ibm-rhel8-s390x-1, test-ibm-ubuntu2204_sharedlibs_container-x64-3, test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/56421/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56468/
Example
not ok 1743 parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
  ---
  duration_ms: 120016.23500
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-assert-esm-cjs-message-verify
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51138/, https://github.com/nodejs/node/pull/51255/, https://github.com/nodejs/node/pull/51168/)
Appeared test-azure_msft-win2016_vs2017-x64-2, test-azure_msft-win11_vs2022-arm64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/56484/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56512/
Example
not ok 69 parallel/test-assert-esm-cjs-message-verify
  ---
  duration_ms: 427.00200
  severity: fail
  exitcode: 1
  stack: |-
    TAP version 13
    # Subtest: ensure the assert.ok throwing similar error messages for esm and cjs files
        # Subtest: should return code 1 for each command
        ok 1 - should return code 1 for each command
          ---
          duration_ms: 182.1517
          ...


Reason parallel/test-inspector-async-stack-traces-set-interval
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51244/, https://github.com/nodejs/node/pull/51168/, https://github.com/nodejs/node/pull/51112/)
Appeared test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/56446/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56506/
Example
not ok 2012 parallel/test-inspector-async-stack-traces-set-interval
  ---
  duration_ms: 15327.02400
  severity: fail
  exitcode: 1
  stack: |-
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:46527/b947eef8-2dd4-4ca2-bf5e-5354c756f620
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
    [test] Skipping the first breakpoint in the eval script
    Timed out waiting for matching notification (break on [eval]:0)
    1
  ...


Reason async-hooks/test-improper-unwind
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/48033/, https://github.com/nodejs/node/pull/50982/)
Appeared test-osuosl-aix72-ppc64_be-3, test-osuosl-aix72-ppc64_be-2
First CI https://ci.nodejs.org/job/node-test-pull-request/56485/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56498/
Example
not ok 65 async-hooks/test-improper-unwind
  ---
  duration_ms: 600.42900
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    
    null !== 1
    
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/async-hooks/test-improper-unwind.js:59:12)
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/common/index.js:476:15)
        at ChildProcess.emit (node:events:519:28)
        at maybeClose (node:internal/child_process:1105:16)
        at Socket.<anonymous> (node:internal/child_process:457:11)
        at Socket.emit (node:events:519:28)
        at Pipe.<anonymous> (node:net:337:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: null,
      expected: 1,
      operator: 'strictEqual'
    }
    
    Node.js v22.0.0-pre
  ...


Reason parallel/test-fs-watch-recursive-add-file-to-new-folder
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51243/, https://github.com/nodejs/node/pull/51255/)
Appeared test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/56423/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56457/
Example
not ok 1106 parallel/test-fs-watch-recursive-add-file-to-new-folder
  ---
  duration_ms: 120041.54400
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason parallel/test-inspector-stop-profile-after-done
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/51108/)
Appeared test-ibm-rhel8-s390x-4, test-ibm-rhel8-s390x-2
First CI https://ci.nodejs.org/job/node-test-pull-request/56460/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56473/
Example
not ok 3554 parallel/test-inspector-stop-profile-after-done
  ---
  duration_ms: 120071.95300
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:43619/4531e88c-8f7f-4c29-a969-85e65113362e
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
  ...


Reason wpt/test-streams
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/50282/, https://github.com/nodejs/node/pull/51264/)
Appeared test-rackspace-win2022_vs2022-x64-6, test-rackspace-win2019_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/56483/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56504/
Example
not ok 965 wpt/test-streams
  ---
  duration_ms: 2779.03200
  severity: fail
  exitcode: 3221226356
  stack: |-
    [SKIPPED] idlharness-shadowrealm.window.js: ShadowRealm support is not enabled
    [SKIPPED] queuing-strategies-size-function-per-global.window.js: Browser-specific test
    [SKIPPED] readable-streams\cross-realm-crash.window.js: Browser-specific test
    [SKIPPED] readable-streams\owning-type-video-frame.any.js: Browser-specific test
    [SKIPPED] readable-streams\read-task-handling.window.js: Browser-specific test
    [SKIPPED] transferable\deserialize-error.window.js: Browser-specific test
    [SKIPPED] transferable\transfer-with-messageport.window.js: Browser-specific test
    [SKIPPED] transform-streams\invalid-realm.tentative.window.js: Browser-specific test
    [PASS] a signal argument 'null' should cause pipeTo() to reject
    [PASS] a signal argument 'AbortSignal' should cause pipeTo() to reject
    [PASS] a signal argument 'true' should cause pipeTo() to reject
    [PASS] a signal argu...

Jenkins Failure
Git Failure
Build Failure
Reason error: The last gc run reported the following. Please correct the root cause
Type BUILD_FAILURE
Failed PR 13 (https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/51246/, https://github.com/nodejs/node/pull/51179/, https://github.com/nodejs/node/pull/51034/, https://github.com/nodejs/node/pull/51138/, https://github.com/nodejs/node/pull/51255/, https://github.com/nodejs/node/pull/51257/, https://github.com/nodejs/node/pull/51254/, https://github.com/nodejs/node/pull/51223/, https://github.com/nodejs/node/pull/50677/, https://github.com/nodejs/node/pull/51112/, https://github.com/nodejs/node/pull/51264/, https://github.com/nodejs/node/pull/51278/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-ibm-ubuntu1804-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/56435/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56511/
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Reason error: unrecognized command line option '-std=gnu++20'; did you mean '-std=gnu++2a'?
Type BUILD_FAILURE
Failed PR 12 (https://github.com/nodejs/node/pull/51246/, https://github.com/nodejs/node/pull/51179/, https://github.com/nodejs/node/pull/51034/, https://github.com/nodejs/node/pull/51138/, https://github.com/nodejs/node/pull/51255/, https://github.com/nodejs/node/pull/51257/, https://github.com/nodejs/node/pull/51254/, https://github.com/nodejs/node/pull/51223/, https://github.com/nodejs/node/pull/50677/, https://github.com/nodejs/node/pull/51112/, https://github.com/nodejs/node/pull/51264/, https://github.com/nodejs/node/pull/51278/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-ubuntu1804_sharedlibs_container-x64-10, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/56442/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56511/
Example
error: unrecognized command line option '-std=gnu++20'; did you mean '-std=gnu++2a'?
deps/simdutf/simdutf.target.mk:86: recipe for target '/home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/obj.target/simdutf/deps/simdutf/simdutf.o' failed
make[2]: *** [/home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/obj.target/simdutf/deps/simdutf/simdutf.o] Error 1
make[2]: *** Waiting for unfinished jobs....
Makefile:134: recipe for target 'node' failed
make[1]: *** [node] Error 2

Reason fatal: No rebase in progress?
Type BUILD_FAILURE
Failed PR 10 (https://github.com/nodejs/node/pull/51124/, https://github.com/nodejs/node/pull/51179/, https://github.com/nodejs/node/pull/51246/, https://github.com/nodejs/node/pull/51034/, https://github.com/nodejs/node/pull/51256/, https://github.com/nodejs/node/pull/51254/, https://github.com/nodejs/node/pull/51223/, https://github.com/nodejs/node/pull/50677/, https://github.com/nodejs/node/pull/51112/, https://github.com/nodejs/node/pull/51278/)
Appeared test-ibm-ubuntu2204_sharedlibs_container-x64-1, test-ibm-ubuntu2204_sharedlibs_container-x64-4, test-ibm-ubuntu2204_sharedlibs_container-x64-3, test-digitalocean-ubuntu2204_sharedlibs_container-x64-5, test-ibm-ubuntu2204_sharedlibs_container-x64-5, test-orka-macos11-x64-1, test-digitalocean-ubuntu2204_sharedlibs_container-x64-7, test-orka-macos11-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/56418/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56511/
Example
fatal: No rebase in progress?

Reason error: unrecognized command line option ‘-std=gnu++20’; did you mean ‘-std=gnu++2a’?
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51257/, https://github.com/nodejs/node/pull/51180/, https://github.com/nodejs/node/pull/51112/)
Appeared test-equinix_mnx-ubuntu1804-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/56468/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56502/
Example
error: unrecognized command line option ‘-std=gnu++20’; did you mean ‘-std=gnu++2a’?
deps/simdutf/simdutf.target.mk:84: recipe for target '/home/iojs/build/workspace/node-test-commit-linux/out/Release/obj.target/simdutf/deps/simdutf/simdutf.o' failed
make[2]: *** [/home/iojs/build/workspace/node-test-commit-linux/out/Release/obj.target/simdutf/deps/simdutf/simdutf.o] Error 1
Makefile:134: recipe for target 'node' failed
make[1]: *** [node] Error 2
Makefile:565: recipe for target 'build-ci' failed

Reason fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/51124/, https://github.com/nodejs/node/pull/50782/, https://github.com/nodejs/node/pull/50454/)
Appeared test-equinix-ubuntu2004_container-armv7l-2, test-equinix-debian11_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/56418/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56451/
Example
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.
fatal: failed to run repack

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/51124/, https://github.com/nodejs/node/pull/50677/)
Appeared test-equinix_mnx-smartos20-x64-3, test-osuosl-aix72-ppc64_be-3, test-osuosl-debian11_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/56425/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56478/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason Failed to trigger sub builds
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/50115/, https://github.com/nodejs/node/pull/50581/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/56434/
Last CI https://ci.nodejs.org/job/node-test-pull-request/56489/
Example
Failed to trigger sub builds

Progress
  • parallel/test-fs-watch-recursive-add-file-to-existing-subfolder (4)
  • parallel/test-assert-esm-cjs-message-verify (3)
  • parallel/test-inspector-async-stack-traces-set-interval (3)
  • async-hooks/test-improper-unwind (2)
  • parallel/test-fs-watch-recursive-add-file-to-new-folder (2)
  • parallel/test-inspector-stop-profile-after-done (2)
  • wpt/test-streams (2)
  • error: The last gc run reported the following. Please correct the root cause (13)
  • error: unrecognized command line option '-std=gnu++20'; did you mean '-std=gnu++2a'? (12)
  • fatal: No rebase in progress? (10)
  • error: unrecognized command line option ‘-std=gnu++20’; did you mean ‘-std=gnu++2a’? (3)
  • fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository (3)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (2)
  • Failed to trigger sub builds (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the listed failing tests, including parallel/test-fs-watch-recursive-add-file-to-existing-subfolder and async-hooks/test-improper-unwind, then inspect their linked CI console logs across the reported platforms. Compare the failures to determine which are reproducible and separate test issues from infrastructure failures. Done means the reported recurring CI failures have identified root causes and the affected tests or CI jobs are reliable again.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.