nodejs / nodejs/reliability

CI Reliability 2022-09-09

Open
#372 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
57
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Failures in node-test-pull-request/46396 to node-test-pull-request/46494 that failed more than 2 PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)

UTC Time RUNNING SUCCESS UNSTABLE ABORTED FAILURE Green Rate
2022-09-09 00:18 0 6 27 8 59 6.52%
JSTest Failure
Reason parallel/test-watch-mode
Type JS_TEST_FAILURE
Failed PR 5 (https://github.com/nodejs/node/pull/44366/, https://github.com/nodejs/node/pull/44247/, https://github.com/nodejs/node/pull/44358/, https://github.com/nodejs/node/pull/44536/, https://github.com/nodejs/node/pull/44565/)
Appeared test-orka-macos10.15-x64-1, test-orka-macos11-x64-2, test-digitalocean-freebsd12-x64-2, test-softlayer-alpine311_container-x64-1, test-digitalocean-alpine312_container-x64-2, test-rackspace-debian10-x64-1, test-rackspace-fedora32-x64-1, test-digitalocean-fedora34-x64-2, test-ibm-rhel8-x64-1, test-digitalocean-ubuntu1804-x64-1, test-equinix-debian10_container-armv7l-2, test-equinix-ubuntu2004_container-armv7l-2, test-osuosl-rhel8-ppc64_le-4, test-softlayer-ubi81_container-x64-1, test-softlayer-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-6, test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-digitalocean-ubuntu1804_sharedlibs_container-x64-3, test-digitalocean-ubuntu1804_sharedlibs_container-x64-4, test-softlayer-ubuntu1804_sharedlibs_container-x64-4, test-digitalocean-ubuntu1804_sharedlibs_container-x64-8, test-equinix-rhel8_container-arm64-1, test-equinix-ubuntu1804_container-arm64-2, test-equinix-ubuntu2004_container-arm64-2, test-equinix-ubuntu2004_container-armv7l-1, test-ibm-rhel8-s390x-4, test-osuosl-aix72-ppc64_be-2, test-nearform-macos11.0-arm64-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-2, test-azure_msft-win10_vs2019-x64-3, test-rackspace-win2012r2_vs2013-x64-1, undefined, test-azure_msft-win10_vs2019-x64-2, test-rackspace-win2012r2_vs2019-x64-5, test-azure_msft-win10_vs2019-x64-4, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/46409/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46472/
Example
not ok 3383 parallel/test-watch-mode
  ---
  duration_ms: 120.82
  severity: fail
  exitcode: -15
  stack: |-
    timeout
    TAP version 13
    # Subtest: watch mode
        # Subtest: should watch changes to a file - event loop ended
        ok 1 - should watch changes to a file - event loop ended
          ---
          duration_ms: 374.953627
          ...


Reason parallel/test-bootstrap-modules
Type JS_TEST_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/44530/, https://github.com/nodejs/node/pull/44358/, https://github.com/nodejs/node/pull/44247/, https://github.com/nodejs/node/pull/44521/)
Appeared undefined
First CI https://ci.nodejs.org/job/node-test-pull-request/46442/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46457/
Example
not ok 400 parallel/test-bootstrap-modules
  ---
  duration_ms: 0.422
  severity: fail
  exitcode: 1
  stack: |-
    (node:31892) internal/test/binding: These APIs are for internal testing only. Do not use them.
    (Use `node --trace-warnings ...` to show where the warning was created)
    
    node:internal/event_target:1006
      process.nextTick(() => { throw err; });
                               ^
    AssertionError [ERR_ASSERTION]: These modules were unexpectedly loaded:
      NativeModule diagnostics_channel
    
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-custom-suites-freestyle/test/parallel/test-bootstrap-modules.js:251:8)
        at Module._compile (node:internal/modules/cjs/loader:1119:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1173:10)
        at Module.load (node:internal/modules/cjs/loader:997:32)
        at Module._load (node:internal/modules/cjs/loader:838:12)
        at Function.executeUserEntryPoint [as runMain] (node:internal/mo...

Reason parallel/test-heapsnapshot-near-heap-limit-worker
Type JS_TEST_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/44498/, https://github.com/nodejs/node/pull/44543/, https://github.com/nodejs/node/pull/44505/)
Appeared test-osuosl-aix72-ppc64_be-3, test-orka-macos11-x64-1, test-ibm-rhel8-s390x-1
First CI https://ci.nodejs.org/job/node-test-pull-request/46427/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46469/
Example
not ok 1227 parallel/test-heapsnapshot-near-heap-limit-worker
  ---
  duration_ms: 5.298
  severity: fail
  exitcode: 1
  stack: |-
    
    Invoked NearHeapLimitCallback, processing=false, current_limit=52428800, initial_limit=52428800
    max_young_gen_size=50331648, young_gen_size=0, old_gen_size=50937216, total_size=50937216
    Estimated available memory=13400801280, estimated overhead=50331648
    Start generating Heap.20220907.204627.5964232.1.001.heapsnapshot...
    Wrote snapshot to /home/iojs/build/.tmp.1185/Heap.20220907.204627.5964232.1.001.heapsnapshot
    
    <--- Last few GCs --->
    
    [5964232:11080f1b0]     3955 ms: Mark-sweep (reduce) 48.5 (52.0) -> 48.5 (52.0) MB, 6.5 / 0.0 ms  (average mu = 0.406, current mu = -0.054) heap profiler; GC in old space requested
    
    
    <--- JS stacktrace --->
    
    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
    
    node:assert:400
        throw err;
        ^
    
    AssertionError ...

Reason report/test-report-fatalerror-oomerror-not-set
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/43315/, https://github.com/nodejs/node/pull/44575/)
Appeared test-joyent-smartos20-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/46489/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46492/
Example
not ok 788 report/test-report-fatalerror-oomerror-not-set
  ---
  duration_ms: 300.133
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason report/test-report-fatalerror-oomerror-set
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/43315/, https://github.com/nodejs/node/pull/44575/)
Appeared test-joyent-smartos20-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/46489/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46492/
Example
not ok 789 report/test-report-fatalerror-oomerror-set
  ---
  duration_ms: 300.119
  severity: fail
  exitcode: -15
  stack: |-
    timeout
  ...


Reason wpt/test-webcrypto
Type JS_TEST_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44500/, https://github.com/nodejs/node/pull/44327/)
Appeared test-azure_msft-win10_vs2019-x64-1
First CI https://ci.nodejs.org/job/node-test-pull-request/46396/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46401/
Example
not ok 898 wpt/test-webcrypto
  ---
  duration_ms: 720.980
  severity: fail
  exitcode: 1
  stack: |-
    timeout
    ---- algorithm-discards-context.https.window.js ----
    [SKIPPED] Not relevant in Node.js context
    ---- historical.any.js ----
    [SKIPPED] Not relevant in Node.js context
    ---- idlharness.https.any.js ----
    [SKIPPED] Various non-IDL-compliant things
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] setup - define tests
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] X25519 good parameters
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] X25519 mixed case parameters
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] X25519 with null length
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] X25519 short result
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    [PASS] X25519 non-multiple of 8 bits
    ---- derive_bits_keys\cfrg_curves_bits.https.any.js ----
    ...

Jenkins Failure
Reason Backing channel 'JNLP4-connect connection from ... is disconnected.
Type JENKINS_FAILURE
Failed PR 7 (https://github.com/nodejs/node/pull/44327/, https://github.com/nodejs/node/pull/44480/, https://github.com/nodejs/node/pull/44479/, https://github.com/nodejs/node/pull/44427/, https://github.com/nodejs/node/pull/44412/, https://github.com/nodejs/node/pull/44247/, https://github.com/nodejs/node/pull/44420/)
Appeared test-digitalocean-freebsd12-x64-2, test-equinix-ubuntu2004_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/46401/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46474/
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 107.170.28.213/107.170.28.213:43943' is disconnected.
	at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
	at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
	at jdk.proxy2/jdk.proxy2.$Proxy87.isAlive(Unknown Source)
	at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
	at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)

Reason Build timed out (after 15 minutes). Marking the build as failed.
Type JENKINS_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44247/, https://github.com/nodejs/node/pull/44521/)
Appeared test-joyent-smartos20-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/46422/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46471/
Example
Build timed out (after 15 minutes). Marking the build as failed.

Build Failure
Reason ERROR: Build step failed with exception
Type BUILD_FAILURE
Failed PR 6 (https://github.com/nodejs/node/pull/44500/, https://github.com/nodejs/node/pull/44509/, https://github.com/nodejs/node/pull/44366/, https://github.com/nodejs/node/pull/44387/, https://github.com/nodejs/node/pull/44392/, https://github.com/nodejs/node/pull/44496/)
Appeared test-equinix-ubuntu2004_container-armv7l-2
First CI https://ci.nodejs.org/job/node-test-pull-request/46396/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46423/
Example
ERROR: Build step failed with exception
java.lang.NullPointerException: no workspace from node hudson.slaves.DumbSlave[test-equinix-ubuntu2004_container-armv7l-2] which is computer hudson.slaves.SlaveComputer@4a73d9df and has channel null
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:114)
	at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
	at org.jenkinsci.plugins.postbuildscript.processor.Processor.processBuildSteps(Processor.java:180)
	at org.jenkinsci.plugins.postbuildscript.processor.Processor.processScripts(Processor.java:91)

Reason dtrace: failed to compile script src/v8ustack.d: Preprocessor failed to process input program
Type BUILD_FAILURE
Failed PR 4 (https://github.com/nodejs/node/pull/44530/, https://github.com/nodejs/node/pull/44247/, https://github.com/nodejs/node/pull/44358/, https://github.com/nodejs/node/pull/44521/)
Appeared test-joyent-smartos20-x64-3, test-joyent-smartos20-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/46442/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46447/
Example
dtrace: failed to compile script src/v8ustack.d: Preprocessor failed to process input program

Reason error: remote GitHub already exists.
Type BUILD_FAILURE
Failed PR 3 (https://github.com/nodejs/node/pull/44524/, https://github.com/nodejs/node/pull/44488/, https://github.com/nodejs/node/pull/44412/)
Appeared test-rackspace-win2012r2_vs2017-x64-1, test-rackspace-win2012r2_vs2019-x64-4
First CI https://ci.nodejs.org/job/node-test-pull-request/46433/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46458/
Example
error: remote GitHub already exists.

> git fetch --prune --no-tags GitHub 

> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp

Reason ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44412/, https://github.com/nodejs/node/pull/44376/)
Appeared test-equinix-debian10_container-armv7l-2, test-equinix-ubuntu2004_container-armv7l-2, test-equinix-debian10_container-armv7l-1, test-equinix-ubuntu1804_sharedlibs_container-arm64-3, test-osuosl-aix72-ppc64_be-3, test-equinix-rhel8_container-arm64-1, test-equinix-ubuntu1804_container-arm64-2, test-equinix-ubuntu2004_container-arm64-2, test-equinix-ubuntu2004_container-armv7l-1
First CI https://ci.nodejs.org/job/node-test-pull-request/46404/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46467/
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
[PostBuildScript] - [INFO] Executing post build scripts.
[node-test-binary-armv7l] $ /bin/bash -ex /tmp/jenkins16479718837328816378.sh
+ '[' -d .git ']'
+ git clean -fdx

Reason Error: module '_bootlocale' has no attribute 'getpreferredencoding'
Type BUILD_FAILURE
Failed PR 2 (https://github.com/nodejs/node/pull/44412/, https://github.com/nodejs/node/pull/44524/)
Appeared test-rackspace-win2012r2_vs2017-x64-3
First CI https://ci.nodejs.org/job/node-test-pull-request/46430/
Last CI https://ci.nodejs.org/job/node-test-pull-request/46433/
Example
Error: module '_bootlocale' has no attribute 'getpreferredencoding'
  [716] Failed to execute script clcache_main
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(423,5): error MSB6006: "clcache_main.exe" exited with code -1. [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_libplatform.vcxproj]
  http_parser.c
  zlib.vcxproj -> ..\..\out\Release\lib\zlib.lib
  http_parser.vcxproj -> ..\..\out\Release\lib\http_parser.lib

Git Failure
Progress
  • parallel/test-watch-mode (5)
  • parallel/test-bootstrap-modules (4)
  • parallel/test-heapsnapshot-near-heap-limit-worker (3)
  • report/test-report-fatalerror-oomerror-not-set (2)
  • report/test-report-fatalerror-oomerror-set (2)
  • wpt/test-webcrypto (2)
  • Backing channel 'JNLP4-connect connection from ... is disconnected. (7)
  • Build timed out (after 15 minutes). Marking the build as failed. (2)
  • ERROR: Build step failed with exception (6)
  • dtrace: failed to compile script src/v8ustack.d: Preprocessor failed to process input program (4)
  • error: remote GitHub already exists. (3)
  • ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? (2)
  • Error: module '_bootlocale' has no attribute 'getpreferredencoding' (2)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the listed CI console links and the failing suites parallel/test-watch-mode, parallel/test-bootstrap-modules, parallel/test-heapsnapshot-near-heap-limit-worker, the report tests, and wpt/test-webcrypto. Compare the failures across the named platforms and rerun the affected CI jobs; done means the recurring failures are understood and the relevant jobs pass reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.