nodejs / nodejs/node-core-utils
`ncu-ci`: output a summary list with all failing tests
オープン
まだ誰も着手していません。
enhancement
- 主要言語
- JavaScript
- スター
- 314
- フォーク
- 137
- 平均マージ
- 2日 8時間
- マージ済み PR（30日）
- 24
説明
So it's easier to know which tests are failing across all platforms. Something like:
------------------------------- Failures Summary -------------------------------
Build failures:
- win-vs2017
- cross-compiler-armv7-gcc-6
- osx1011
Test failures:
- sequential/test-inspector-break-when-eval (4 platforms)
- sequential/test-perf-hooks (2 platforms)
- parallel/test-memory-usage (1 platform)
- parallel/test-repl-history-navigation (1 platform)
- sequential/test-heap-prof (1 platform)
- sequential/test-worker-prof (1 platform)
Example:
$ ncu-ci url ci.nodejs.org/job/node-test-pull-request/29794
--------------------------------------------------------------------------------
[1/1] Running PR: 29597
--------------------------------------------------------------------------------
✔ Build data downloaded
✔ Data downloaded
----------------------------------- Summary ------------------------------------
Result FAILURE
URL https://ci.nodejs.org/job/node-test-pull-request/29597/
Source https://github.com/nodejs/node/pull/32116/
Commit [1ab2b73b513c] test: warn when inspector process crashes
Date 2020-03-06 16:25:26 -0800
Author Matheus Marchini <mmarchini@netflix.com>
------------------------------- Failures Summary -------------------------------
Build failures:
- win-vs2017
- cross-compiler-armv7-gcc-6
- osx1011
Test failures:
- sequential/test-inspector-break-when-eval (4 platforms)
- sequential/test-perf-hooks (2 platforms)
- parallel/test-memory-usage (1 platform)
- parallel/test-repl-history-navigation (1 platform)
- sequential/test-heap-prof (1 platform)
- sequential/test-worker-prof (1 platform)
----------------------------------- osx1011 ------------------------------------
URL https://ci.nodejs.org/job/node-test-commit-osx/nodes=osx1011/32407/console
Type BUILD_FAILURE
Built On test-macstadium-macos10.11-x64-1
Reason fatal: No rebase in progress?
----------------------------------- osx1015 ------------------------------------
URL https://ci.nodejs.org/job/node-test-commit-osx/nodes=osx1015/32407/console
Type JS_TEST_FAILURE
Built On test-nearform-macos10.15-x64-3
Reason
not ok 2777 sequential/test-inspector-break-when-eval
---
duration_ms: 15.390
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:63158/870bb3cd-ad48-4622-8a87-53091991bef8
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[test] Setting up a debugger
[err] Debugger attached.
[err]
[test] Breaking in the code
[err] child process crashed, signal SIGSEGV
Timed out waiting for matching notification (break on file:///Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/inspector-global-function.js:9))
1
...
------------------------------- ppcle-ubuntu1404 -------------------------------
URL https://ci.nodejs.org/job/node-test-commit-plinux/nodes=ppcle-ubuntu1404/31338/console
Type JS_TEST_FAILURE
Built On test-osuosl-ubuntu1404-ppc64_le-2
Reason
not ok 2792 sequential/test-inspector-break-when-eval
---
duration_ms: 15.341
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:52357/f0bedc84-050f-4b9e-a641-286ac55dcb28
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[test] Setting up a debugger
[err] Debugger attached.
[err]
[test] Breaking in the code
[err] child process crashed, signal SIGSEGV
Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/fixtures/inspector-global-function.js:9))
1
...
--------------------------------- smartos18-64 ---------------------------------
URL https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/31307/console
Type JS_TEST_FAILURE
Built On test-joyent-smartos18-x64-3
Reason
not ok 2847 sequential/test-inspector-break-when-eval
---
duration_ms: 15.745
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:44215/d810e836-4d14-4696-a415-2aa7f92714e5
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[test] Setting up a debugger
[err] Debugger attached.
[err]
[test] Breaking in the code
[err] child process crashed, signal SIGSEGV
Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-smartos/nodes/smartos18-64/test/fixtures/inspector-global-function.js:9))
1
...
--------------------- ubuntu1804_sharedlibs_withoutssl_x64 ---------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_withoutssl_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-1
Reason
not ok 1393 parallel/test-memory-usage
---
duration_ms: 1.351
severity: fail
exitcode: 1
stack: |-
assert.js:384
throw err;
^
AssertionError [ERR_ASSERTION]: 11571790 - 1102879 >= 10485760
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-memory-usage.js:42:3)
at Module._compile (internal/modules/cjs/loader.js:1202:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
at Module.load (internal/modules/cjs/loader.js:1051:32)
at Function.Module._load (internal/modules/cjs/loader.js:947:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
...
--------------------- ubuntu1804_sharedlibs_withoutssl_x64 ---------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_withoutssl_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-1
Reason
not ok 2768 sequential/test-perf-hooks
---
duration_ms: 1.83
severity: fail
exitcode: 1
stack: |-
{
name: 'node',
entryType: 'node',
startTime: 0,
duration: { around: 468.111141204834 },
nodeStart: { around: 0 },
v8Start: { around: 0 },
bootstrapComplete: { around: 457.42479133605957, delay: 2500 },
environment: { around: 0 },
loopStart: -1,
loopExit: -1
}
assert.js:384
throw err;
^
AssertionError [ERR_ASSERTION]: v8Start: 275.37767028808594 >= 250
at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:67:7)
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:79:1)
at Module._compile (internal/modules/cjs/loader.js:1202:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
at Module.load (internal/modules/cjs/loader.js:1051:32)
at Function.Module._load (internal/modules/cjs/loader.js:947:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
...
------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
not ok 1648 parallel/test-repl-history-navigation
---
duration_ms: 12.367
severity: fail
exitcode: 1
stack: |-
assert.js:102
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected ... Lines skipped
[
'\x1B[1G',
...
------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
not ok 2741 sequential/test-heap-prof
---
duration_ms: 120.338
severity: fail
exitcode: -15
stack: |-
timeout
...
------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
not ok 2768 sequential/test-inspector-break-when-eval
---
duration_ms: 6.359
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:43601/97b597e2-1f2f-470a-b4f6-183159499193
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[out] Ready!
[out]
[test] Setting up a debugger
[err] Debugger attached.
[err]
[test] Breaking in the code
[test] Step over console statement and test output
[out] 0 3
[out]
[test] Verify node waits for the frontend to disconnect
[err] Waiting for the debugger to disconnect...
[err]
[err] internal/timers.js:404
[err] node.priorityQueuePosition = pos;
[err] ^
[err]
[err] TypeError: Cannot set property 'priorityQueuePosition' of undefined
[err] at setPosition (internal/timers.js:404:30)
[err] at PriorityQueue.percolateDown (internal/priority_queue.js:70:7)
[err] at listOnTimeout (internal/timers.js:512:24)
[err] at processTimers (internal/timers.js:492:7)
[err]
AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
1 !== 0
at runTests (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-inspector-break-when-eval.js:65:10)
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
generatedMessage: true,
code: 'ERR_ASSERTION',
actual: 1,
expected: 0,
operator: 'strictEqual'
}
1
...
------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
not ok 2802 sequential/test-perf-hooks
---
duration_ms: 0.788
severity: fail
exitcode: 1
stack: |-
{
name: 'node',
entryType: 'node',
startTime: 0,
duration: { around: 426.24277687072754 },
nodeStart: { around: 0 },
v8Start: { around: 0 },
bootstrapComplete: { around: 421.70141792297363, delay: 2500 },
environment: { around: 0 },
loopStart: -1,
loopExit: -1
}
assert.js:384
throw err;
^
AssertionError [ERR_ASSERTION]: v8Start: 250.21231079101562 >= 250
at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:67:7)
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:79:1)
at Module._compile (internal/modules/cjs/loader.js:1202:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
at Module.load (internal/modules/cjs/loader.js:1051:32)
at Function.Module._load (internal/modules/cjs/loader.js:947:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: false,
expected: true,
operator: '=='
}
...
------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type JS_TEST_FAILURE
Built On test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
not ok 2821 sequential/test-worker-prof
---
duration_ms: 31.410
severity: fail
exitcode: 1
stack: |-
assert.js:102
throw new AssertionError(obj);
^
AssertionError [ERR_ASSERTION]: child exited with signal: {
error: Error: spawnSync /home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node ETIMEDOUT
at Object.spawnSync (internal/child_process.js:1070:20)
at spawnSync (child_process.js:597:24)
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js:51:23)
at Module._compile (internal/modules/cjs/loader.js:1202:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
at Module.load (internal/modules/cjs/loader.js:1051:32)
at Function.Module._load (internal/modules/cjs/loader.js:947:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'spawnSync /home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node',
path: '/home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node',
spawnargs: [
'--prof',
'/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js',
'child'
]
},
status: null,
signal: 'SIGTERM',
output: [ null, '', '' ],
pid: 4629,
stdout: '',
stderr: ''
}
at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js:57:10)
at Module._compile (internal/modules/cjs/loader.js:1202:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
at Module.load (internal/modules/cjs/loader.js:1051:32)
at Function.Module._load (internal/modules/cjs/loader.js:947:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
generatedMessage: false,
code: 'ERR_ASSERTION',
actual: 'SIGTERM',
expected: null,
operator: 'strictEqual'
}
...
------------------------------ centos7-arm64-gcc6 ------------------------------
URL https://ci.nodejs.org/job/node-test-commit-arm/nodes=centos7-arm64-gcc6/29838/console
Type JS_TEST_FAILURE
Built On test-packetnet-centos7-arm64-2
Reason
not ok 2866 sequential/test-inspector-break-when-eval
---
duration_ms: 15.633
severity: fail
exitcode: 1
stack: |-
[test] Connecting to a child Node process
[test] Testing /json/list
[err] Debugger listening on ws://127.0.0.1:41871/5e35f89c-2fda-4abb-ac8f-9234ebdca44c
[err] For help, see: https://nodejs.org/en/docs/inspector
[err]
[test] Setting up a debugger
[err] Debugger attached.
[err]
[test] Breaking in the code
[err] child process crashed, signal SIGSEGV
Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-arm/nodes/centos7-arm64-gcc6/test/fixtures/inspector-global-function.js:9))
1
...
---------------------------------- win-vs2017 ----------------------------------
URL https://ci.nodejs.org/job/node-compile-windows/nodes=win-vs2017/32291/console
Type BUILD_FAILURE
Built On test-rackspace-win2012r2_vs2017-x64-4
Reason
error C2139: 'v8::internal::SmallOrderedHashSet': an undefined class is not allowed as an argument to compiler intrinsic type trait '__is_convertible_to' [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
c:\workspace\node-compile-windows\node\out\release\obj\global_intermediate\torque-output-root\torque-generated\class-definitions-tq.h(151): note: see declaration of 'v8::internal::SmallOrderedHashSet'
c:\workspace\node-compile-windows\node\deps\v8\src\codegen\tnode.h(250): note: see reference to class template instantiation 'std::is_convertible<T,v8::internal::Object>' being compiled
with
[
-------------------------- cross-compiler-armv7-gcc-6 --------------------------
URL https://ci.nodejs.org/job/node-cross-compile/nodes=cross-compiler-armv7-gcc-6/28127/console
Type BUILD_FAILURE
Built On test-joyent-ubuntu1604_arm_cross-x64-1
Reason
error: unable to delete 'jenkins-node-test-commit-arm-fanned-1ab2b73b513cc7aa3908c0bd10f1c8238607a4e4-binary-pi1p/cc-armv7': remote ref does not exist
error: failed to push some refs to 'binary_tmp@169.60.150.88:binary_tmp.git'
+ true
+ JOBS=2
+ CONFIG_FLAGS=' --dest-cpu=arm'
+ make build-ci -j2
--------------------------------- Other builds ---------------------------------
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
ncu-ci コマンドと既存のサマリー出力から始め、プラットフォーム間でビルドとテストの失敗がどのように収集されるかを追跡します。出力に、例の形式に従って、グループ化されたビルドの失敗とテストの失敗が含まれ、各テストのプラットフォーム数が示されれば、作業は完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- javascript
- 領域
- ci-cd, cli
- issue の種類
- 機能追加
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100