nodejs / nodejs/reliability

CI Reliability 2023-08-30

Open
#650 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/53531 to node-test-pull-request/53630 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2023-08-30 00:12 0 0 19 2 79 0.00%
Jenkins Failure
Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/49327/, https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49155/)
Appeared test-rackspace-win2019_vs2019-x64-2, test-digitalocean-ubi81_container-x64-1, test-rackspace-win2022_vs2022-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53608/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53626/
Example
Build timed out (after 60 minutes). Marking the build as failed.

CCTest Failure
Reason Value of: (!expectation.read_expected)
Type CC_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/48919/, https://github.com/nodejs/node/pull/49185/, https://github.com/nodejs/node/pull/49150/)
Appeared test-digitalocean-alpine318_container-x64-1, test-digitalocean-alpine315_container-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53594/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53627/
Example
Value of: (!expectation.read_expected)
  Actual: true
Expected: false

[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (5000 ms)

Reason C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected)
Type CC_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49400/)
Appeared test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/53550/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53606/
Example
C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected)
  Actual: true
Expected: false

[  FAILED  ] InspectorSocketTest.HostIPv6NonRoutable (5001 ms)

Build Failure
Reason Failed to trigger fanned build
Type BUILD_FAILURE
Failed PR 21 (https://github.com/nodejs/node/pull/49313/, https://github.com/nodejs/node/pull/48244/, https://github.com/nodejs/node/pull/49193/, https://github.com/nodejs/node/pull/49030/, https://github.com/nodejs/node/pull/49151/, https://github.com/nodejs/node/pull/49137/, https://github.com/nodejs/node/pull/48970/, https://github.com/nodejs/node/pull/49341/, https://github.com/nodejs/node/pull/49155/, https://github.com/nodejs/node/pull/49335/, https://github.com/nodejs/node/pull/49320/, https://github.com/nodejs/node/pull/49274/, https://github.com/nodejs/node/pull/49348/, https://github.com/nodejs/node/pull/49346/, https://github.com/nodejs/node/pull/48919/, https://github.com/nodejs/node/pull/49239/, https://github.com/nodejs/node/pull/49161/, https://github.com/nodejs/node/pull/49185/, https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49238/, https://github.com/nodejs/node/pull/49340/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/53555/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53602/
Example
Failed to trigger fanned build

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/48607/, https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/48244/, https://github.com/nodejs/node/pull/49185/, https://github.com/nodejs/node/pull/49220/)
Appeared test-equinix-rhel8_container-arm64-2, test-equinix-ubuntu1804_container-arm64-2, test-equinix-ubuntu2004_sharedlibs_container-arm64-4, test-equinix-ubuntu2004_container-armv7l-2, test-equinix_mnx-smartos20-x64-3, test-equinix-ubuntu2004_sharedlibs_container-arm64-3, test-osuosl-aix72-ppc64_be-2, test-osuosl-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_container-armv7l-1, test-equinix_mnx-smartos20-x64-4, test-osuosl-aix72-ppc64_be-4, test-equinix-ubuntu1804_sharedlibs_container-arm64-5, test-equinix-ubuntu2004_sharedlibs_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53546/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53630/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason ERROR: No files found with the specified search criteria.
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49161/)
Appeared test-rackspace-win2012r2_vs2019-x64-2, test-rackspace-win2022_vs2022-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/53552/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53559/
Example
ERROR: No files found with the specified search criteria.
        1 file(s) copied.

> set DEBUG_HELPER=1 

> call vcbuild.bat arm64 release msi

Reason ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/48607/, https://github.com/nodejs/node/pull/49220/)
Appeared test-digitalocean-freebsd12-x64-2, test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-3
First CI https://ci.nodejs.org/job/node-test-pull-request/53546/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53630/
Example
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Collecting metadata...
Metadata collection done.
Notifying upstream projects of job completion
Finished: FAILURE

Reason fatal: bad object refs/heads/jenkins-node-test-commit-arm-fanned-44627007be7eb3934e39bdd3df7e8f2556a2cd44-binary-pi1p/cc-armv7
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49161/)
Appeared test-rackspace-win2019_vs2019-x64-3, test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2022_vs2022-x64-3, test-rackspace-win2022_vs2022-x64-1, test-rackspace-win2022_vs2022-x64-5
First CI https://ci.nodejs.org/job/node-test-pull-request/53552/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53559/
Example
fatal: bad object refs/heads/jenkins-node-test-commit-arm-fanned-44627007be7eb3934e39bdd3df7e8f2556a2cd44-binary-pi1p/cc-armv7        
fatal: bad object refs/heads/jenkins-node-test-commit-arm-fanned-44627007be7eb3934e39bdd3df7e8f2556a2cd44-binary-pi1p/cc-armv7

JSTest Failure
Reason sequential/test-single-executable-application-snapshot-and-code-cache
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/49221/, https://github.com/nodejs/node/pull/49155/, https://github.com/nodejs/node/pull/49274/, https://github.com/nodejs/node/pull/49400/, https://github.com/nodejs/node/pull/49401/)
Appeared test-digitalocean-ubuntu1804_sharedlibs_container-x64-2, test-equinix-ubuntu2004_container-arm64-2, test-rackspace-debian10-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/53543/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53617/
Example
not ok 3975 sequential/test-single-executable-application-snapshot-and-code-cache
  ---
  duration_ms: 10665.40800
  severity: fail
  exitcode: 1
  stack: |-
    warning: Can't find string offset for section name '.note'
    node:assert:125
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    + actual - expected
    
    + ''
    - 'Hello from snapshot'
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-single-executable-application-snapshot-and-code-cache.js:62:10)
        at Module._compile (node:internal/modules/cjs/loader:1241:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
        at Module.load (node:internal/modules/cjs/loader:1091:32)
        at Module._load (node:internal/modules/cjs/loader:938:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
        at node:internal/main/run_main_mod...

Reason node-api/test_threadsafe_function/test
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/49185/, https://github.com/nodejs/node/pull/49341/, https://github.com/nodejs/node/pull/49401/)
Appeared test-equinix-ubuntu2004_container-armv7l-2, test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53605/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53617/
Example
not ok 140 node-api/test_threadsafe_function/test
  ---
  duration_ms: 360043.49900
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason sequential/test-cli-syntax-require
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/49238/, https://github.com/nodejs/node/pull/49274/, https://github.com/nodejs/node/pull/49150/)
Appeared test-ibm-aix72-ppc64_be-2, test-ibm-ubi81_container-x64-1, test-osuosl-aix72-ppc64_be-3
First CI https://ci.nodejs.org/job/node-test-pull-request/53577/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53623/
Example
not ok 3887 sequential/test-cli-syntax-require
  ---
  duration_ms: 464.61600
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:1026
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The input did not match the regular expression /^SyntaxError: \b/m. Input:
    
    ''
    
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/sequential/test-cli-syntax-require.js:34:14)
        at ChildProcess.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/common/index.js:474:15)
        at ChildProcess.emit (node:events:514:28)
        at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
      generatedMessage: true,
      code: 'ERR_ASSERTION',
      actual: '',
      expected: /^SyntaxError: \b/m,
      operator: 'match'
    }
    
    Node.js v21.0.0-pre
  ...


Reason parallel/test-inspector-stop-profile-after-done
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49346/, https://github.com/nodejs/node/pull/49150/)
Appeared test-ibm-rhel8-s390x-4, test-ibm-rhel8-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53590/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53627/
Example
not ok 3602 parallel/test-inspector-stop-profile-after-done
  ---
  duration_ms: 120091.52600
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:37875/52bb99aa-e8c6-42ea-aca9-0b90febec763
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] 
  ...


Reason parallel/test-runner-watch-mode
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49401/, https://github.com/nodejs/node/pull/49155/)
Appeared test-equinix-ubuntu1804_sharedlibs_container-arm64-1, test-osuosl-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53617/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53626/
Example
not ok 3646 parallel/test-runner-watch-mode
  ---
  duration_ms: 360085.64700
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    TAP version 13
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    Updating /home/iojs/node-tmp/.tmp.2317/dependent.js
    # Subtest: test runner watch mode
        # Subtest: should run tests repeatedly
        ok 1 - should run tests repeatedly
          ---
          duration_ms: 488.047525
          ...


Reason sequential/test-http-regr-gh-2928
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49238/, https://github.com/nodejs/node/pull/49150/)
Appeared test-digitalocean-freebsd12-x64-1, test-azure_msft-win2016_vs2017-x64-6
First CI https://ci.nodejs.org/job/node-test-pull-request/53603/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53623/
Example
not ok 3905 sequential/test-http-regr-gh-2928
  ---
  duration_ms: 120114.43200
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+....

Reason sequential/test-perf-hooks
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49346/)
Appeared undefined, test-ibm-rhel7-s390x-1
First CI https://ci.nodejs.org/job/node-test-pull-request/53558/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53590/
Example
not ok 3684 sequential/test-perf-hooks
  ---
  duration_ms: 183.28500
  severity: fail
  exitcode: 1
  stack: |-
    check initial nodeTiming
    {
      "name": "node",
      "entryType": "node",
      "startTime": 0,
      "duration": 93.68861198425293,
      "nodeStart": 1.408529281616211,
      "v8Start": 6.066031455993652,
      "environment": 42.85482978820801,
      "loopStart": 63.976563453674316,
      "loopExit": -1,
      "bootstrapComplete": 63.952603340148926,
      "idleTime": 0
    }
    
    node:internal/event_target:1079
      process.nextTick(() => { throw err; });
                               ^
    AssertionError [ERR_ASSERTION]: now (108.07290363311768) - timing.duration (93.68861198425293) = 14.384291648864746 >= 10
        at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/sequential/test-perf-hooks.js:104:3)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/sequential/test-perf-hooks.js:11...

Reason sequential/test-worker-prof
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/49150/, https://github.com/nodejs/node/pull/49346/)
Appeared test-macstadium-macos11.0-arm64-3, test-azure_msft-win2022_vs2019-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53548/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53590/
Example
not ok 3979 sequential/test-worker-prof
  ---
  duration_ms: 498.41400
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:399
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: 14 >= 15
        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/sequential/test-worker-prof.js:77:5)
        at Module._compile (node:internal/modules/cjs/loader:1241:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
        at Module.load (node:internal/modules/cjs/loader:1091:32)
        at Module._load (node:internal/modules/cjs/loader:938:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12)
        at node:internal/main/run_main_module:23:47 {
      generatedMessage: false,
      code: 'ERR_ASSERTION',
      actual: false,
      expected: true,
      operator: '=='
    }
    
    Node.js v21.0.0-pre
  ...


Progress
  • Build timed out (after 60 minutes). Marking the build as failed. (3)
  • Value of: (!expectation.read_expected) (3)
  • C:\workspace\node-compile-windows\node\test\cctest\test_inspector_socket.cc(323): error: Value of: (!expectation.read_expected) (2)
  • Failed to trigger fanned build (21)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (5)
  • ERROR: No files found with the specified search criteria. (2)
  • ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error? (2)
  • fatal: bad object refs/heads/jenkins-node-test-commit-arm-fanned-44627007be7eb3934e39bdd3df7e8f2556a2cd44-binary-pi1p/cc-armv7 (2)
  • sequential/test-single-executable-application-snapshot-and-code-cache (5)
  • node-api/test_threadsafe_function/test (3)
  • sequential/test-cli-syntax-require (3)
  • parallel/test-inspector-stop-profile-after-done (2)
  • parallel/test-runner-watch-mode (2)
  • sequential/test-http-regr-gh-2928 (2)
  • sequential/test-perf-hooks (2)
  • sequential/test-worker-prof (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ncu-ci walk pr --stats=true report and inspect the linked Jenkins job logs for the recurring failure groups. Pay particular attention to test/cctest/test_inspector_socket.cc and the listed sequential JavaScript test; done means the affected CI jobs no longer reproduce these failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell
Domain
ci-cd, devops, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.