nodejs / nodejs/reliability

CI Reliability 2022-04-27

Open
#264 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/43618 to node-test-pull-request/43711 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2022-04-27 00:14 4 5 27 2 62 5.32%
Jenkins Failure
Reason Build timed out (after 60 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42837/, https://github.com/nodejs/node/pull/42747/)
Appeared test-softlayer-ubi81_container-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/43655/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43701/
Example
Build timed out (after 60 minutes). Marking the build as failed.

Build Failure
Reason error C2338: sizeof(MemoryChunk) <= MemoryChunk::kHeaderSize [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_init.vcxproj]
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/42725/, https://github.com/nodejs/node/pull/42754/, https://github.com/nodejs/node/pull/42782/)
Appeared test-rackspace-win2012r2_vs2019-x64-1, test-rackspace-win2012r2_vs2019-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/43635/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43707/
Example
error C2338: sizeof(MemoryChunk) <= MemoryChunk::kHeaderSize [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_init.vcxproj]
C:\workspace\node-compile-windows\node\deps\v8\src\heap\spaces.h(321,1): error C2338: sizeof(Page) <= MemoryChunk::kHeaderSize [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_init.vcxproj]
C:\workspace\node-compile-windows\node\deps\v8\src\heap\large-spaces.h(56,1): error C2338: sizeof(LargePage) <= MemoryChunk::kHeaderSize [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_init.vcxproj]
  node_js2c
  v8_pch.cc

undefined
Reason Unknown
Type undefined
Failed PR 10 (https://github.com/nodejs/node/pull/42776/, https://github.com/nodejs/node/pull/42784/, https://github.com/nodejs/node/pull/42761/, https://github.com/nodejs/node/pull/42701/, https://github.com/nodejs/node/pull/42799/, https://github.com/nodejs/node/pull/42834/, https://github.com/nodejs/node/pull/42837/, https://github.com/nodejs/node/pull/42847/, https://github.com/nodejs/node/pull/42872/, https://github.com/nodejs/node/pull/42865/)
Appeared test-nearform_arm-win10_vs2019-arm64-1, test-equinix-debian10_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/43633/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43705/
Example
Unknown

JSTest Failure
Reason parallel/test-http-server-request-timeout-keepalive
Type JS_TEST_FAILURE
Failed PR 17 (https://github.com/nodejs/node/pull/42803/, https://github.com/nodejs/node/pull/42820/, https://github.com/nodejs/node/pull/42809/, https://github.com/nodejs/node/pull/42815/, https://github.com/nodejs/node/pull/42768/, https://github.com/nodejs/node/pull/42782/, https://github.com/nodejs/node/pull/42837/, https://github.com/nodejs/node/pull/42852/, https://github.com/nodejs/node/pull/42818/, https://github.com/nodejs/node/pull/42863/, https://github.com/nodejs/node/pull/42867/, https://github.com/nodejs/node/pull/42616/, https://github.com/nodejs/node/pull/42725/, https://github.com/nodejs/node/pull/42761/, https://github.com/nodejs/node/pull/42875/, https://github.com/nodejs/node/pull/42747/, https://github.com/nodejs/node/pull/42849/)
Appeared test-macstadium-macos11.0-arm64-3, test-orka-macos11-x64-2, test-digitalocean-freebsd12-x64-2, test-digitalocean-freebsd12-x64-1, test-macstadium-macos11.0-arm64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/43618/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43710/
Example
not ok 1370 parallel/test-http-server-request-timeout-keepalive
  ---
  duration_ms: 6.498
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:123
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    + actual - expected
    
    + 'HTTP/1.1 200 OK\r\n' +
    +   'Content-Type: text/plain\r\n' +
    +   'Date: Tue, 26 Apr 2022 16:39:47 GMT\r\n' +
    +   'Connection: close\r\n' +
    +   'Transfer-Encoding: chunked\r\n' +
    +   '\r\n' +
    +   '0\r\n' +
    +   '\r\n'
    - ''
        at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/parallel/test-http-server-request-timeout-keepalive.js:79:12)
        at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/common/index.js:438:15)
        at Socket.emit (node:events:539:35)
        at endReadableNT (node:internal/streams/readable:1344:12)
        at process.processTicksAndRejections (node:internal/...

Reason parallel/test-http-server-headers-timeout-keepalive
Type JS_TEST_FAILURE
Failed PR 16 (https://github.com/nodejs/node/pull/42776/, https://github.com/nodejs/node/pull/42807/, https://github.com/nodejs/node/pull/42701/, https://github.com/nodejs/node/pull/42815/, https://github.com/nodejs/node/pull/42761/, https://github.com/nodejs/node/pull/42799/, https://github.com/nodejs/node/pull/42849/, https://github.com/nodejs/node/pull/42766/, https://github.com/nodejs/node/pull/42852/, https://github.com/nodejs/node/pull/42845/, https://github.com/nodejs/node/pull/42857/, https://github.com/nodejs/node/pull/42818/, https://github.com/nodejs/node/pull/42616/, https://github.com/nodejs/node/pull/42863/, https://github.com/nodejs/node/pull/42872/, https://github.com/nodejs/node/pull/42725/)
Appeared test-macstadium-macos11.0-arm64-3, test-orka-macos11-x64-2, test-digitalocean-freebsd12-x64-1, test-nearform-macos10.15-x64-1, test-digitalocean-freebsd12-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/43621/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43706/
Example
not ok 1304 parallel/test-http-server-headers-timeout-keepalive
  ---
  duration_ms: 6.420
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:123
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
    + actual - expected
    
    + 'HTTP/1.1 200 OK\r\n' +
    +   'Content-Type: text/plain\r\n' +
    +   'Date: Tue, 26 Apr 2022 13:02:41 GMT\r\n' +
    +   'Connection: close\r\n' +
    +   'Transfer-Encoding: chunked\r\n' +
    +   '\r\n' +
    +   '0\r\n' +
    +   '\r\n'
    - ''
        at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/parallel/test-http-server-headers-timeout-keepalive.js:81:12)
        at Socket.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/test/common/index.js:438:15)
        at Socket.emit (node:events:539:35)
        at endReadableNT (node:internal/streams/readable:1344:12)
        at process.processTicksAndRejections (node:internal/...

Reason parallel/test-trace-events-fs-sync
Type JS_TEST_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/42776/, https://github.com/nodejs/node/pull/42784/, https://github.com/nodejs/node/pull/42761/, https://github.com/nodejs/node/pull/42701/, https://github.com/nodejs/node/pull/42799/, https://github.com/nodejs/node/pull/42834/, https://github.com/nodejs/node/pull/42865/)
Appeared test-azure_msft-win10_vs2019-x64-2, test-azure_msft-win10_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/43633/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43705/
Example
not ok 698 parallel/test-trace-events-fs-sync
  ---
  duration_ms: 2.31
  severity: fail
  exitcode: 1
  stack: |-
    C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-trace-events-fs-sync.js:133
        throw new Error(`${tr}:\n${util.inspect(proc)}`);
        ^
    
    Error: fs.sync.fstat:
    {
      status: 1,
      signal: null,
      output: [
        null,
        '',
        'node:internal/fs/utils:345\r\n' +
          '    throw err;\r\n' +
          '    ^\r\n' +
          '\r\n' +
          "Error: EPERM: operation not permitted, open 'fs.txt'\r\n" +
          '    at Object.openSync (node:fs:590:3)\r\n' +
          '    at Object.writeFileSync (node:fs:2184:35)\r\n' +
          '    at [eval]:1:4\r\n' +
          '    at Script.runInThisContext (node:vm:129:12)\r\n' +
          '    at Object.runInThisContext (node:vm:305:38)\r\n' +
          '    at node:internal/process/execution:76:19\r\n' +
          '    at [eval]-wrapper:6:22\r\n' +
          '    at evalScript (node...

Reason parallel/test-gc-http-client-timeout
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/42834/, https://github.com/nodejs/node/pull/42842/, https://github.com/nodejs/node/pull/42846/, https://github.com/nodejs/node/pull/42852/, https://github.com/nodejs/node/pull/42869/)
Appeared test-digitalocean-freebsd12-x64-1, test-digitalocean-freebsd12-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/43648/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43711/
Example
not ok 1132 parallel/test-gc-http-client-timeout
  ---
  duration_ms: 0.435
  severity: fail
  exitcode: 1
  stack: |-
    We should do 300 requests
    Done: 0/300
    Collected: 0/231
    node:events:505
          throw er; // Unhandled 'error' event
          ^
    
    Error: connect ECONNRESET 127.0.0.1:30468 - Local (127.0.0.1:30662)
        at internalConnect (node:net:959:16)
        at defaultTriggerAsyncIdScope (node:internal/async_hooks:463:18)
        at GetAddrInfoReqWrap.emitLookup [as callback] (node:net:1105:9)
        at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:85:8)
    Emitted 'error' event on ClientRequest instance at:
        at Socket.socketErrorListener (node:_http_client:454:9)
        at Socket.emit (node:events:527:28)
        at emitErrorNT (node:internal/streams/destroy:151:8)
        at emitErrorCloseNT (node:internal/streams/destroy:116:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      errno: -54,
      code: 'ECONNRESE...

Reason parallel/test-inspector-multisession-ws
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/42784/, https://github.com/nodejs/node/pull/42725/, https://github.com/nodejs/node/pull/42616/)
Appeared test-nearform-macos10.15-x64-1, test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/43636/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43688/
Example
not ok 1587 parallel/test-inspector-multisession-ws
  ---
  duration_ms: 15.530
  severity: fail
  exitcode: 1
  stack: |-
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [test] Connecting to a child Node process
    [test] Testing /json/list
    [err] Debugger listening on ws://127.0.0.1:64237/f340d693-06d9-4eda-a882-b245c3a138d5
    [err] For help, see: https://nodejs.org/en/docs/inspector
    [err] 
    [err] Debugger attached.
    [err] Debugger attached.
    [err] 
    [test] Breaking in code and verifying events are fired
    Timed out waiting for matching notification (Initial pause)
    1
  ...


Reason parallel/test-heapsnapshot-near-heap-limit-worker
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42849/, https://github.com/nodejs/node/pull/42872/)
Appeared test-ibm-rhel8-s390x-4
First CI https://ci.nodejs.org/job/node-test-pull-request/43663/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43709/
Example
not ok 965 parallel/test-heapsnapshot-near-heap-limit-worker
  ---
  duration_ms: 2.842
  severity: fail
  exitcode: 1
  stack: |-
    
    Invoked NearHeapLimitCallback, processing=false, current_limit=52428800, initial_limit=52428800
    max_young_gen_size=50331648, young_gen_size=0, old_gen_size=49845424, total_size=49845424
    Estimated available memory=9223372036749512704, estimated overhead=50331648
    Start generating Heap.20220426.120555.1859065.1.001.heapsnapshot...
    Wrote snapshot to /home/iojs/node-tmp/.tmp.945/Heap.20220426.120555.1859065.1.001.heapsnapshot
    
    <--- Last few GCs --->
    
    [1859065:0x3ffa4000d20]     1485 ms: Mark-sweep (reduce) 47.5 (55.1) -> 47.5 (52.1) MB, 14.3 / 0.0 ms  (average mu = 0.384, current mu = 0.471) heap profiler; GC in old space requested
    [1859065:0x3ffa4000d20]     1504 ms: Mark-sweep (reduce) 47.5 (52.1) -> 47.5 (52.1) MB, 19.2 / 0.0 ms  (average mu = 0.207, current mu = 0.001) heap profiler; GC in old space requested
    
    
    <--- JS stackt...

Reason parallel/test-runner-cli
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42776/, https://github.com/nodejs/node/pull/42725/)
Appeared test-nearform-macos10.15-x64-1, test-nearform-macos10.15-x64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/43621/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43640/
Example
not ok 2022 parallel/test-runner-cli
  ---
  duration_ms: 0.737
  severity: fail
  exitcode: 1
  stack: |-
    node:assert:1026
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: The input did not match the regular expression /not ok 1 - this should fail/. Input:
    
    'TAP version 13\n' +
      'not ok 1 - /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/test-runner/index.js\n' +
      '  ---\n' +
      '  duration_ms: 0.05715329\n' +
      "  failureType: 'subtestsFailed'\n" +
      '  exitCode: 1\n' +
      "  stdout: ''\n" +
      '  stderr: |-\n' +
      '    /Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/test-runner/index.js:2\n' +
      "    throw new Error('thrown from index.js');\n" +
      '    ^\n' +
      '    \n' +
      '    Error: thrown from index.js\n' +
      '        at Object.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/test-runner/index.js:2:7)\n' +
      '        a...

Reason parallel/test-worker-heap-snapshot
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/42761/, https://github.com/nodejs/node/pull/42799/)
Appeared test-rackspace-win2012r2_vs2013-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/43642/
Last CI https://ci.nodejs.org/job/node-test-pull-request/43646/
Example
not ok 773 parallel/test-worker-heap-snapshot
  ---
  duration_ms: 1.362
  severity: fail
  exitcode: 134
  stack: |-
    FATAL ERROR: HandleScope::HandleScope Entering the V8 API without proper locking in place
  ...


Progress
  • Build timed out (after 60 minutes). Marking the build as failed. (2)
  • error C2338: sizeof(MemoryChunk) <= MemoryChunk::kHeaderSize [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_init.vcxproj] (3)
  • Unknown (10)
  • parallel/test-http-server-request-timeout-keepalive (17)
  • parallel/test-http-server-headers-timeout-keepalive (16)
  • parallel/test-trace-events-fs-sync (7)
  • parallel/test-gc-http-client-timeout (5)
  • parallel/test-inspector-multisession-ws (3)
  • parallel/test-heapsnapshot-near-heap-limit-worker (2)
  • parallel/test-runner-cli (2)
  • parallel/test-worker-heap-snapshot (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated reliability report and the linked Jenkins job logs, especially the listed parallel test files such as test/parallel/test-http-server-request-timeout-keepalive.js and test/parallel/test-trace-events-fs-sync.js. Compare the failures across the named platforms and determine which CI or test issue needs correction; done would require a reproducible diagnosis and reliable CI results, but this issue does not specify a particular fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell
Domain
build-system, ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.