nodejs / nodejs/node-core-utils

`ncu-ci`: output a summary list with all failing tests

Đang mở
#396 2 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

enhancement
Ngôn ngữ chính
JavaScript
Star
314
Fork
137
Merge trung bình
2 ngày 8 giờ
Pull request đã merge (30 ngày)
24

Mô tả

So it's easier to know which tests are failing across all platforms. Something like:

------------------------------- Failures Summary -------------------------------
Build failures:
  - win-vs2017
  - cross-compiler-armv7-gcc-6
  - osx1011
Test failures:
  - sequential/test-inspector-break-when-eval (4 platforms)
  - sequential/test-perf-hooks (2 platforms)
  - parallel/test-memory-usage (1 platform)
  - parallel/test-repl-history-navigation (1 platform)
  - sequential/test-heap-prof (1 platform)
  - sequential/test-worker-prof (1 platform)

Example:

$ ncu-ci url ci.nodejs.org/job/node-test-pull-request/29794
--------------------------------------------------------------------------------
[1/1] Running PR: 29597
--------------------------------------------------------------------------------
✔  Build data downloaded
✔  Data downloaded
----------------------------------- Summary ------------------------------------
Result     FAILURE
URL        https://ci.nodejs.org/job/node-test-pull-request/29597/
Source     https://github.com/nodejs/node/pull/32116/
Commit     [1ab2b73b513c] test: warn when inspector process crashes
Date       2020-03-06 16:25:26 -0800
Author     Matheus Marchini <mmarchini@netflix.com>
------------------------------- Failures Summary -------------------------------
Build failures:
  - win-vs2017
  - cross-compiler-armv7-gcc-6
  - osx1011
Test failures:
  - sequential/test-inspector-break-when-eval (4 platforms)
  - sequential/test-perf-hooks (2 platforms)
  - parallel/test-memory-usage (1 platform)
  - parallel/test-repl-history-navigation (1 platform)
  - sequential/test-heap-prof (1 platform)
  - sequential/test-worker-prof (1 platform)
----------------------------------- osx1011 ------------------------------------
URL        https://ci.nodejs.org/job/node-test-commit-osx/nodes=osx1011/32407/console
Type       BUILD_FAILURE
Built On   test-macstadium-macos10.11-x64-1
Reason     fatal: No rebase in progress?
----------------------------------- osx1015 ------------------------------------
URL        https://ci.nodejs.org/job/node-test-commit-osx/nodes=osx1015/32407/console
Type       JS_TEST_FAILURE
Built On   test-nearform-macos10.15-x64-3
Reason
  not ok 2777 sequential/test-inspector-break-when-eval
    ---
    duration_ms: 15.390
    severity: fail
    exitcode: 1
    stack: |-
      [test] Connecting to a child Node process
      [test] Testing /json/list
      [err] Debugger listening on ws://127.0.0.1:63158/870bb3cd-ad48-4622-8a87-53091991bef8
      [err] For help, see: https://nodejs.org/en/docs/inspector
      [err]
      [test] Setting up a debugger
      [err] Debugger attached.
      [err]
      [test] Breaking in the code
      [err] child process crashed, signal SIGSEGV
      Timed out waiting for matching notification (break on file:///Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1015/test/fixtures/inspector-global-function.js:9))
      1
    ...

------------------------------- ppcle-ubuntu1404 -------------------------------
URL        https://ci.nodejs.org/job/node-test-commit-plinux/nodes=ppcle-ubuntu1404/31338/console
Type       JS_TEST_FAILURE
Built On   test-osuosl-ubuntu1404-ppc64_le-2
Reason
  not ok 2792 sequential/test-inspector-break-when-eval
    ---
    duration_ms: 15.341
    severity: fail
    exitcode: 1
    stack: |-
      [test] Connecting to a child Node process
      [test] Testing /json/list
      [err] Debugger listening on ws://127.0.0.1:52357/f0bedc84-050f-4b9e-a641-286ac55dcb28
      [err] For help, see: https://nodejs.org/en/docs/inspector
      [err]
      [test] Setting up a debugger
      [err] Debugger attached.
      [err]
      [test] Breaking in the code
      [err] child process crashed, signal SIGSEGV
      Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/fixtures/inspector-global-function.js:9))
      1
    ...

--------------------------------- smartos18-64 ---------------------------------
URL        https://ci.nodejs.org/job/node-test-commit-smartos/nodes=smartos18-64/31307/console
Type       JS_TEST_FAILURE
Built On   test-joyent-smartos18-x64-3
Reason
  not ok 2847 sequential/test-inspector-break-when-eval
    ---
    duration_ms: 15.745
    severity: fail
    exitcode: 1
    stack: |-
      [test] Connecting to a child Node process
      [test] Testing /json/list
      [err] Debugger listening on ws://127.0.0.1:44215/d810e836-4d14-4696-a415-2aa7f92714e5
      [err] For help, see: https://nodejs.org/en/docs/inspector
      [err]
      [test] Setting up a debugger
      [err] Debugger attached.
      [err]
      [test] Breaking in the code
      [err] child process crashed, signal SIGSEGV
      Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-smartos/nodes/smartos18-64/test/fixtures/inspector-global-function.js:9))
      1
    ...

--------------------- ubuntu1804_sharedlibs_withoutssl_x64 ---------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_withoutssl_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-1
Reason
  not ok 1393 parallel/test-memory-usage
    ---
    duration_ms: 1.351
    severity: fail
    exitcode: 1
    stack: |-
      assert.js:384
          throw err;
          ^

      AssertionError [ERR_ASSERTION]: 11571790 - 1102879 >= 10485760
          at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-memory-usage.js:42:3)
          at Module._compile (internal/modules/cjs/loader.js:1202:30)
          at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
          at Module.load (internal/modules/cjs/loader.js:1051:32)
          at Function.Module._load (internal/modules/cjs/loader.js:947:14)
          at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
          at internal/main/run_main_module.js:17:47 {
        generatedMessage: false,
        code: 'ERR_ASSERTION',
        actual: false,
        expected: true,
        operator: '=='
      }
    ...

--------------------- ubuntu1804_sharedlibs_withoutssl_x64 ---------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_withoutssl_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-1
Reason
  not ok 2768 sequential/test-perf-hooks
    ---
    duration_ms: 1.83
    severity: fail
    exitcode: 1
    stack: |-
      {
        name: 'node',
        entryType: 'node',
        startTime: 0,
        duration: { around: 468.111141204834 },
        nodeStart: { around: 0 },
        v8Start: { around: 0 },
        bootstrapComplete: { around: 457.42479133605957, delay: 2500 },
        environment: { around: 0 },
        loopStart: -1,
        loopExit: -1
      }
      assert.js:384
          throw err;
          ^

      AssertionError [ERR_ASSERTION]: v8Start: 275.37767028808594 >= 250
          at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:67:7)
          at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:79:1)
          at Module._compile (internal/modules/cjs/loader.js:1202:30)
          at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
          at Module.load (internal/modules/cjs/loader.js:1051:32)
          at Function.Module._load (internal/modules/cjs/loader.js:947:14)
          at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
          at internal/main/run_main_module.js:17:47 {
        generatedMessage: false,
        code: 'ERR_ASSERTION',
        actual: false,
        expected: true,
        operator: '=='
      }
    ...

------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
  not ok 1648 parallel/test-repl-history-navigation
    ---
    duration_ms: 12.367
    severity: fail
    exitcode: 1
    stack: |-
      assert.js:102
        throw new AssertionError(obj);
        ^

      AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
      + actual - expected ... Lines skipped

        [
          '\x1B[1G',
      ...

------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
  not ok 2741 sequential/test-heap-prof
    ---
    duration_ms: 120.338
    severity: fail
    exitcode: -15
    stack: |-
      timeout
    ...

------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
  not ok 2768 sequential/test-inspector-break-when-eval
    ---
    duration_ms: 6.359
    severity: fail
    exitcode: 1
    stack: |-
      [test] Connecting to a child Node process
      [test] Testing /json/list
      [err] Debugger listening on ws://127.0.0.1:43601/97b597e2-1f2f-470a-b4f6-183159499193
      [err] For help, see: https://nodejs.org/en/docs/inspector
      [err]
      [out] Ready!
      [out]
      [test] Setting up a debugger
      [err] Debugger attached.
      [err]
      [test] Breaking in the code
      [test] Step over console statement and test output
      [out] 0 3
      [out]
      [test] Verify node waits for the frontend to disconnect
      [err] Waiting for the debugger to disconnect...
      [err]
      [err] internal/timers.js:404
      [err]   node.priorityQueuePosition = pos;
      [err]                              ^
      [err]
      [err] TypeError: Cannot set property 'priorityQueuePosition' of undefined
      [err]     at setPosition (internal/timers.js:404:30)
      [err]     at PriorityQueue.percolateDown (internal/priority_queue.js:70:7)
      [err]     at listOnTimeout (internal/timers.js:512:24)
      [err]     at processTimers (internal/timers.js:492:7)
      [err]
      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

      1 !== 0

          at runTests (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-inspector-break-when-eval.js:65:10)
          at processTicksAndRejections (internal/process/task_queues.js:97:5) {
        generatedMessage: true,
        code: 'ERR_ASSERTION',
        actual: 1,
        expected: 0,
        operator: 'strictEqual'
      }
      1
    ...

------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
  not ok 2802 sequential/test-perf-hooks
    ---
    duration_ms: 0.788
    severity: fail
    exitcode: 1
    stack: |-
      {
        name: 'node',
        entryType: 'node',
        startTime: 0,
        duration: { around: 426.24277687072754 },
        nodeStart: { around: 0 },
        v8Start: { around: 0 },
        bootstrapComplete: { around: 421.70141792297363, delay: 2500 },
        environment: { around: 0 },
        loopStart: -1,
        loopExit: -1
      }
      assert.js:384
          throw err;
          ^

      AssertionError [ERR_ASSERTION]: v8Start: 250.21231079101562 >= 250
          at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:67:7)
          at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-perf-hooks.js:79:1)
          at Module._compile (internal/modules/cjs/loader.js:1202:30)
          at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
          at Module.load (internal/modules/cjs/loader.js:1051:32)
          at Function.Module._load (internal/modules/cjs/loader.js:947:14)
          at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
          at internal/main/run_main_module.js:17:47 {
        generatedMessage: false,
        code: 'ERR_ASSERTION',
        actual: false,
        expected: true,
        operator: '=='
      }
    ...

------------------------ ubuntu1804_sharedlibs_zlib_x64 ------------------------
URL        https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_zlib_x64/18497/console
Type       JS_TEST_FAILURE
Built On   test-joyent-ubuntu1804_sharedlibs_container-x64-2
Reason
  not ok 2821 sequential/test-worker-prof
    ---
    duration_ms: 31.410
    severity: fail
    exitcode: 1
    stack: |-
      assert.js:102
        throw new AssertionError(obj);
        ^

      AssertionError [ERR_ASSERTION]: child exited with signal: {
        error: Error: spawnSync /home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node ETIMEDOUT
            at Object.spawnSync (internal/child_process.js:1070:20)
            at spawnSync (child_process.js:597:24)
            at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js:51:23)
            at Module._compile (internal/modules/cjs/loader.js:1202:30)
            at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
            at Module.load (internal/modules/cjs/loader.js:1051:32)
            at Function.Module._load (internal/modules/cjs/loader.js:947:14)
            at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
            at internal/main/run_main_module.js:17:47 {
          errno: -110,
          code: 'ETIMEDOUT',
          syscall: 'spawnSync /home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node',
          path: '/home/iojs/build/workspace/node-test-commit-linux-containered/out/Release/node',
          spawnargs: [
            '--prof',
            '/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js',
            'child'
          ]
        },
        status: null,
        signal: 'SIGTERM',
        output: [ null, '', '' ],
        pid: 4629,
        stdout: '',
        stderr: ''
      }
          at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/sequential/test-worker-prof.js:57:10)
          at Module._compile (internal/modules/cjs/loader.js:1202:30)
          at Object.Module._extensions..js (internal/modules/cjs/loader.js:1222:10)
          at Module.load (internal/modules/cjs/loader.js:1051:32)
          at Function.Module._load (internal/modules/cjs/loader.js:947:14)
          at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
          at internal/main/run_main_module.js:17:47 {
        generatedMessage: false,
        code: 'ERR_ASSERTION',
        actual: 'SIGTERM',
        expected: null,
        operator: 'strictEqual'
      }
    ...

------------------------------ centos7-arm64-gcc6 ------------------------------
URL        https://ci.nodejs.org/job/node-test-commit-arm/nodes=centos7-arm64-gcc6/29838/console
Type       JS_TEST_FAILURE
Built On   test-packetnet-centos7-arm64-2
Reason
  not ok 2866 sequential/test-inspector-break-when-eval
    ---
    duration_ms: 15.633
    severity: fail
    exitcode: 1
    stack: |-
      [test] Connecting to a child Node process
      [test] Testing /json/list
      [err] Debugger listening on ws://127.0.0.1:41871/5e35f89c-2fda-4abb-ac8f-9234ebdca44c
      [err] For help, see: https://nodejs.org/en/docs/inspector
      [err]
      [test] Setting up a debugger
      [err] Debugger attached.
      [err]
      [test] Breaking in the code
      [err] child process crashed, signal SIGSEGV
      Timed out waiting for matching notification (break on file:///home/iojs/build/workspace/node-test-commit-arm/nodes/centos7-arm64-gcc6/test/fixtures/inspector-global-function.js:9))
      1
    ...

---------------------------------- win-vs2017 ----------------------------------
URL        https://ci.nodejs.org/job/node-compile-windows/nodes=win-vs2017/32291/console
Type       BUILD_FAILURE
Built On   test-rackspace-win2012r2_vs2017-x64-4
Reason
  error C2139: 'v8::internal::SmallOrderedHashSet': an undefined class is not allowed as an argument to compiler intrinsic type trait '__is_convertible_to' [C:\workspace\node-compile-windows\node\tools\v8_gypfiles\v8_initializers.vcxproj]
    c:\workspace\node-compile-windows\node\out\release\obj\global_intermediate\torque-output-root\torque-generated\class-definitions-tq.h(151): note: see declaration of 'v8::internal::SmallOrderedHashSet'
    c:\workspace\node-compile-windows\node\deps\v8\src\codegen\tnode.h(250): note: see reference to class template instantiation 'std::is_convertible<T,v8::internal::Object>' being compiled
            with
            [
-------------------------- cross-compiler-armv7-gcc-6 --------------------------
URL        https://ci.nodejs.org/job/node-cross-compile/nodes=cross-compiler-armv7-gcc-6/28127/console
Type       BUILD_FAILURE
Built On   test-joyent-ubuntu1604_arm_cross-x64-1
Reason
  error: unable to delete 'jenkins-node-test-commit-arm-fanned-1ab2b73b513cc7aa3908c0bd10f1c8238607a4e4-binary-pi1p/cc-armv7': remote ref does not exist
  error: failed to push some refs to 'binary_tmp@169.60.150.88:binary_tmp.git'
  + true
  + JOBS=2
  + CONFIG_FLAGS=' --dest-cpu=arm'
  + make build-ci -j2
--------------------------------- Other builds ---------------------------------

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Hướng nghiên cứu

Bắt đầu từ lệnh ncu-ci và đầu ra tóm tắt hiện có của lệnh đó, sau đó truy vết cách các lỗi build và test được thu thập trên các nền tảng. Công việc hoàn tất khi đầu ra bao gồm các lỗi build và test được nhóm lại, cùng với số lượng nền tảng của từng test, theo đúng định dạng ví dụ.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
javascript
Lĩnh vực
ci-cd, cli
Loại issue
Tính năng
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Đình trệ
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.