CI Reliability 2023-11-24
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 25/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- javascript, node.js, shell
- 领域
- ci-cd, devops, testing-qa
调研方向
从链接的 CI 控制台日志开始,查看 parallel/test-worker-nearheaplimit-deadlock、parallel/test-fs-watch-recursive-add-file-to-existing-subfolder 以及列出的其他测试,并比较它们在所列平台上的失败情况。使用 ncu-ci walk pr --stats=true 报告的上下文来识别可复现的模式。当报告的 CI 失败已得到理解,并且受影响的测试在列出的环境中不再间歇性失败时,即视为完成。
由索引模型根据 Issue 内容生成。
描述
Failures in node-test-pull-request/55784 to node-test-pull-request/55882 that failed 2 or more PRs
(Generated with ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)
| UTC Time | RUNNING | SUCCESS | UNSTABLE | ABORTED | FAILURE | Green Rate |
|---|---|---|---|---|---|---|
| 2023-11-24 00:13 | 0 | 14 | 17 | 0 | 69 | 14.00% |
JSTest Failure
Example
not ok 3262 parallel/test-worker-nearheaplimit-deadlock
---
duration_ms: 173.04800
severity: crashed
exitcode: -6
stack: |-
<--- Last few GCs --->
[49547:0x280008000] 28 ms: Mark-Compact 1.7 (6.0) -> 1.7 (6.0) MB, 1.46 / 0.00 ms (average mu = 0.321, current mu = 0.027) background allocation failure; GC in old space requested
[49547:0x280008000] 30 ms: Mark-Compact 1.7 (6.0) -> 1.7 (6.0) MB, 2.12 / 0.00 ms (average mu = 0.197, current mu = 0.019) background allocation failure; GC in old space requested
<--- JS stacktrace --->
FATAL ERROR: LocalHeap: allocation failed Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0x104c97db8 node::Abort() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
2: 0x104c97fb8 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx1...
| Reason | parallel/test-fs-watch-recursive-add-file-to-existing-subfolder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 6 (https://github.com/nodejs/node/pull/50828/, https://github.com/nodejs/node/pull/50715/, https://github.com/nodejs/node/pull/50661/, https://github.com/nodejs/node/pull/49156/, https://github.com/nodejs/node/pull/49884/, https://github.com/nodejs/node/pull/50860/) |
| Appeared | test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55793/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55874/ |
Example
not ok 1365 parallel/test-fs-watch-recursive-add-file-to-existing-subfolder
---
duration_ms: 120112.84500
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-blob-file-backed |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 3 (https://github.com/nodejs/node/pull/50791/, https://github.com/nodejs/node/pull/50825/, https://github.com/nodejs/node/pull/50773/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-9, test-digitalocean-alpine318_container-x64-1, test-digitalocean-ubi81_container-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55784/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55838/ |
Example
not ok 278 parallel/test-blob-file-backed
---
duration_ms: 501.47400
severity: fail
exitcode: 1
stack: |-
node:internal/blob:291
const error = lazyDOMException('The blob could not be read', 'NotReadableError');
^
DOMException [NotReadableError]: The blob could not be read
at BlobReader.<anonymous> (node:internal/blob:291:25)
Node.js v22.0.0-pre
...
| Reason | parallel/test-debugger-run-after-quit-restart |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50661/, https://github.com/nodejs/node/pull/49884/) |
| Appeared | test-ibm-aix72-ppc64_be-1, test-ibm-aix72-ppc64_be-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55801/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55876/ |
Example
not ok 777 parallel/test-debugger-run-after-quit-restart
---
duration_ms: 5464.20400
severity: fail
exitcode: 1
stack: |-
/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/common/debugger.js:90
reject(new Error([
^
Error: Timeout (5000) while waiting for /break (?:on start )?in/i; found:
at Timeout.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix72-ppc64/test/common/debugger.js:90:18)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7)
Node.js v22.0.0-pre
...
| Reason | parallel/test-fs-watch-recursive-symlink |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50759/, https://github.com/nodejs/node/pull/50820/) |
| Appeared | test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55791/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55803/ |
Example
not ok 1369 parallel/test-fs-watch-recursive-symlink
---
duration_ms: 120099.65800
severity: fail
exitcode: -15
stack: |-
timeout
...
| Reason | parallel/test-inspector-scriptparsed-context |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50715/, https://github.com/nodejs/node/pull/50840/) |
| Appeared | test-ibm-ubi81_container-x64-1, test-ibm-rhel8-s390x-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55810/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55844/ |
Example
not ok 2255 parallel/test-inspector-scriptparsed-context
---
duration_ms: 15338.84100
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:35069/5219d0d7-8f76-4233-b8bd-770bf90ff872
[err]
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[err] Debugger attached.
[err]
Timed out waiting for matching notification (break on [eval]:2)
1
...
| Reason | parallel/test-inspector-stop-profile-after-done |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50834/, https://github.com/nodejs/node/pull/50858/) |
| Appeared | test-ibm-rhel8-s390x-1, test-ibm-rhel8-s390x-4 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55850/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55868/ |
Example
not ok 2419 parallel/test-inspector-stop-profile-after-done
---
duration_ms: 120765.27000
severity: fail
exitcode: -15
stack: |-
timeout
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:36693/73fa285e-ca53-4b86-9759-e5498c2ce303
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[err] Debugger attached.
[err]
...
| Reason | parallel/test-runner-inspect |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50828/, https://github.com/nodejs/node/pull/49884/) |
| Appeared | test-ibm-rhel8-s390x-4, test-ibm-rhel8-s390x-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55793/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55849/ |
Example
not ok 3665 parallel/test-runner-inspect
---
duration_ms: 120006.77300
severity: fail
exitcode: -15
stack: |-
timeout
[test] Connecting to a child Node process
[test] Testing /json/list
[err] (node:3432802) Warning: Using the inspector with --test forces running at a concurrency of 1. Use the inspectPort option to run with concurrency
[err] (Use `node --trace-warnings ...` to show where the warning was created)
[err]
[out] TAP version 13
[out]
[err] Debugger listening on ws://127.0.0.1:41245/abefdb08-aac7-4f6c-9b3a-0acdc4622ce8
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[err] Debugger attached.
[err]
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[out] # Debugger ending on ws://127.0.0.1:41245/abefdb08-aac7-4f6c-9b3a-0acdc4622ce8
[out]
...
| Reason | parallel/test-string-decoder |
|---|---|
| Type | JS_TEST_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50825/, https://github.com/nodejs/node/pull/50820/) |
| Appeared | test-rackspace-win2016_vs2015-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55794/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55803/ |
Example
not ok 717 parallel/test-string-decoder
---
duration_ms: 420.99400
severity: fail
exitcode: 1
stack: |-
node:assert:635
throw err;
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected
+ Comparison {}
- Comparison {
- code: 'ERR_STRING_TOO_LONG'
- }
at Object.<anonymous> (C:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-string-decoder.js:205:10)
at Module._compile (node:internal/modules/cjs/loader:1376:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
at Module.load (node:internal/modules/cjs/loader:1207:32)
at Module._load (node:internal/modules/cjs/loader:1023:12)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
at node:internal/main/run_main_module:28:49 {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: RangeError: Array buffer alloc...
Jenkins Failure
| Reason | Backing channel 'JNLP4-connect connection from ... is disconnected. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50730/, https://github.com/nodejs/node/pull/50833/) |
| Appeared | test-digitalocean-fedora38-x64-2, test-orka-macos11-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55813/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55834/ |
Example
java.io.IOException: Backing channel 'JNLP4-connect connection from 134.209.172.40/134.209.172.40:41402' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at jdk.proxy2/jdk.proxy2.$Proxy166.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)
| Reason | Build timed out (after 120 minutes). Marking the build as failed. |
|---|---|
| Type | JENKINS_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50651/, https://github.com/nodejs/node/pull/50758/) |
| Appeared | test-digitalocean-fedora38-x64-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55817/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55870/ |
Example
Build timed out (after 120 minutes). Marking the build as failed.
Git Failure
Build Failure
| Reason | fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 12 (https://github.com/nodejs/node/pull/50787/, https://github.com/nodejs/node/pull/49793/, https://github.com/nodejs/node/pull/50759/, https://github.com/nodejs/node/pull/50825/, https://github.com/nodejs/node/pull/50730/, https://github.com/nodejs/node/pull/50790/, https://github.com/nodejs/node/pull/50709/, https://github.com/nodejs/node/pull/50840/, https://github.com/nodejs/node/pull/50834/, https://github.com/nodejs/node/pull/50846/, https://github.com/nodejs/node/pull/50858/, https://github.com/nodejs/node/pull/50860/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55788/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55881/ |
Example
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository
fatal: Could not read from remote repository.
fatal: failed to run repack
| Reason | ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 8 (https://github.com/nodejs/node/pull/50791/, https://github.com/nodejs/node/pull/50709/, https://github.com/nodejs/node/pull/50651/, https://github.com/nodejs/node/pull/50594/, https://github.com/nodejs/node/pull/48882/, https://github.com/nodejs/node/pull/50763/, https://github.com/nodejs/node/pull/50730/, https://github.com/nodejs/node/pull/50860/) |
| Appeared | test-equinix-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2, test-osuosl-ubuntu2004_container-armv7l-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55784/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55874/ |
Example
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?
[PostBuildScript] - [INFO] Executing post build scripts.
[node-test-binary-armv7l] $ /bin/bash -ex /tmp/jenkins8683146355660303296.sh
+ '[' -d .git ']'
+ git clean -fdx
Removing out/
| Reason | fatal error: Killed signal terminated program cc1plus |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 6 (https://github.com/nodejs/node/pull/50820/, https://github.com/nodejs/node/pull/50831/, https://github.com/nodejs/node/pull/49793/, https://github.com/nodejs/node/pull/50833/, https://github.com/nodejs/node/pull/49846/, https://github.com/nodejs/node/pull/49884/) |
| Appeared | test-digitalocean-rhel8_arm_cross_container-x64-2, test-digitalocean-fedora37-x64-1, test-digitalocean-alpine318_container-x64-1, test-digitalocean-rhel8_arm_cross_container-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55803/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55839/ |
Example
fatal error: Killed signal terminated program cc1plus
| Reason | error: The last gc run reported the following. Please correct the root cause |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/34111/, https://github.com/nodejs/node/pull/50730/) |
| Appeared | test-digitalocean-ubuntu1804_sharedlibs_container-x64-1, test-ibm-ubuntu1804-x64-1 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55798/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55878/ |
Example
error: The last gc run reported the following. Please correct the root cause
and remove .git/gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
| Reason | error: remote GitHub already exists. |
|---|---|
| Type | BUILD_FAILURE |
| Failed PR | 2 (https://github.com/nodejs/node/pull/50354/, https://github.com/nodejs/node/pull/50653/) |
| Appeared | test-rackspace-win2022_vs2022-x64-1, test-rackspace-win2022_vs2022-x64-3, test-rackspace-win2022_vs2022-x64-2, test-rackspace-win2022_vs2022-x64-5 |
| First CI | https://ci.nodejs.org/job/node-test-pull-request/55851/ |
| Last CI | https://ci.nodejs.org/job/node-test-pull-request/55857/ |
Example
error: remote GitHub already exists.
> git fetch --prune --no-tags GitHub
> git fetch --prune --no-tags https://github.com/nodejs/build.git +refs/heads/main:refs/remotes/jenkins_tmp
CCTest Failure
undefined
Example
Unknown
Progress
-
parallel/test-worker-nearheaplimit-deadlock(17) -
parallel/test-fs-watch-recursive-add-file-to-existing-subfolder(6) -
parallel/test-blob-file-backed(3) -
parallel/test-debugger-run-after-quit-restart(2) -
parallel/test-fs-watch-recursive-symlink(2) -
parallel/test-inspector-scriptparsed-context(2) -
parallel/test-inspector-stop-profile-after-done(2) -
parallel/test-runner-inspect(2) -
parallel/test-string-decoder(2) -
Backing channel 'JNLP4-connect connection from ... is disconnected.(2) -
Build timed out (after 120 minutes). Marking the build as failed.(2) -
fatal: '/home/iojs/.ccache/node.shared.reference' does not appear to be a git repository(12) -
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?(8) -
fatal error: Killed signal terminated program cc1plus(6) -
error: The last gc run reported the following. Please correct the root cause(2) -
error: remote GitHub already exists.(2) -
Unknown(11)
- 主要语言
- Shell
- 星标
- 57
- 派生
- 20
- PR 合并指标
- 30 天内没有已合并 PR
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
nodejs/reliability 的其他 Issue
-
难度 4/5 3-5 天 新手友好度 35/100
nodejs/reliability#1666 ·
-
难度 4/5 3-5 天 新手友好度 25/100
nodejs/reliability#1665 ·
-
难度 5/5 一周以上 新手友好度 25/100
nodejs/reliability#1664 ·
-
难度 4/5 3-5 天 新手友好度 35/100
nodejs/reliability#1663 ·
-
难度 4/5 3-5 天 新手友好度 35/100
nodejs/reliability#1662 ·
查看 nodejs/reliability 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
community-scripts/ProxmoxVE#17396 · 1 条评论 ·
-
package-update
难度 2/5 1-3 小时 新手友好度 78/100
oSoWoSo/vOid_Community_repOsitory#144 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
-
help wanted new command
难度 2/5 1-3 小时 新手友好度 72/100
tldr-pages/tldr#24151 ·
-
PacReq: Rune IDE 未关闭package request
难度 2/5 1-3 小时 新手友好度 68/100
pacstall/pacstall-programs#9402 ·