Flaky test-runner-run-watch
Open
Nobody has claimed this yet.
flaky-test
macos
never-stale
windows
- Dominant language
- JavaScript
- Stars
- 122k
- Forks
- 37.3k
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 283
Description
Test
test-runner-run-watch
Platform
Windows
Console output
16:41:02 ok 848 parallel/test-vfs
16:41:02 ---
16:44:04 duration_ms: 1255.99700
16:44:04 ...
16:44:04 failed 10 out of 10
16:44:04 not ok 849 parallel/test-runner-run-watch
16:44:04 ---
16:44:04 duration_ms: 121964.85900
16:44:04 severity: fail
16:44:04 exitcode: 1
16:44:04 stack: |-
16:44:04 timeout
16:44:04 TAP version 13
16:44:04 # Subtest: test runner watch mode
16:44:04 # Subtest: should run tests repeatedly
16:44:04 not ok 1 - should run tests repeatedly
16:44:04 ---
16:44:04 duration_ms: 11682.5747
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\parallel\\test-runner-run-watch.mjs:155:3'
16:44:04 failureType: 'testCodeFailure'
16:44:04 error: |-
16:44:04 The input did not match the regular expression /# tests 1/. Input:
16:44:04
16:44:04 '# Subtest: test has ran\n' +
16:44:04 'ok 1 - test has ran\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 37.95\n' +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 2 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1068.7518\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 3 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1050.696\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 4 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1012.4334\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 5 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 974.3208\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 6 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 999.9954\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 7 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1011.2067\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test has ran\n' +
16:44:04 'ok 8 - test has ran\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 29.1914\n' +
16:44:04 ' ...\n' +
16:44:04 '1..8\n' +
16:44:04 '# tests 8\n' +
16:44:04 '# suites 0\n' +
16:44:04 '# pass 2\n' +
16:44:04 '# fail 6\n' +
16:44:04 '# cancelled 0\n' +
16:44:04 '# skipped 0\n' +
16:44:04 '# todo 0\n' +
16:44:04 '# duration_ms 1693.5803\n'
16:44:04
16:44:04 code: 'ERR_ASSERTION'
16:44:04 name: 'AssertionError'
16:44:04 expected:
16:44:04 actual: |-
16:44:04 # Subtest: test has ran
16:44:04 ok 1 - test has ran
16:44:04 ---
16:44:04 duration_ms: 37.95
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 2 - test.js
16:44:04 ---
16:44:04 duration_ms: 1068.7518
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 3 - test.js
16:44:04 ---
16:44:04 duration_ms: 1050.696
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 4 - test.js
16:44:04 ---
16:44:04 duration_ms: 1012.4334
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 5 - test.js
16:44:04 ---
16:44:04 duration_ms: 974.3208
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 6 - test.js
16:44:04 ---
16:44:04 duration_ms: 999.9954
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 7 - test.js
16:44:04 ---
16:44:04 duration_ms: 1011.2067
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test has ran
16:44:04 ok 8 - test has ran
16:44:04 ---
16:44:04 duration_ms: 29.1914
16:44:04 ...
16:44:04 1..8
16:44:04 # tests 8
16:44:04 # suites 0
16:44:04 # pass 2
16:44:04 # fail 6
16:44:04 # cancelled 0
16:44:04 # skipped 0
16:44:04 # todo 0
16:44:04 # duration_ms 1693.5803
16:44:04
16:44:04 operator: 'match'
16:44:04 stack: |-
16:44:04 testUpdate (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:81:14)
16:44:04 process.processTicksAndRejections (node:internal/process/task_queues:105:5)
16:44:04 async testWatch (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:147:26)
16:44:04 async TestContext.<anonymous> (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:156:5)
16:44:04 async Test.run (node:internal/test_runner/test:879:9)
16:44:04 async Promise.all (index 0)
16:44:04 async Suite.run (node:internal/test_runner/test:1239:7)
16:44:04 async Test.processPendingSubtests (node:internal/test_runner/test:590:7)
16:44:04 ...
16:44:04 # Subtest: should run tests with dependency repeatedly
16:44:04 not ok 2 - should run tests with dependency repeatedly
16:44:04 ---
16:44:04 duration_ms: 9670.1542
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\parallel\\test-runner-run-watch.mjs:159:3'
16:44:04 failureType: 'testCodeFailure'
16:44:04 error: |-
16:44:04 The input did not match the regular expression /# tests 1/. Input:
16:44:04
16:44:04 '# Subtest: test has ran\n' +
16:44:04 'ok 1 - test has ran\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 10.4875\n' +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 2 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1085.3394\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 3 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1111.3339\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 4 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 954.7013\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 5 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 1070.1657\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test.js\n' +
16:44:04 'not ok 6 - test.js\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 926.3438\n' +
16:44:04 " location: 'c:\\\\workspace\\\\node-test-binary-windows-js-suites\\\\node\\\\test\\\\.tmp.682\\\\test.js:1:1'\n" +
16:44:04 " failureType: 'testCodeFailure'\n" +
16:44:04 ' exitCode: ~\n' +
16:44:04 " signal: 'SIGTERM'\n" +
16:44:04 " error: 'test failed'\n" +
16:44:04 " code: 'ERR_TEST_FAILURE'\n" +
16:44:04 ' ...\n' +
16:44:04 '# Subtest: test has ran\n' +
16:44:04 'ok 7 - test has ran\n' +
16:44:04 ' ---\n' +
16:44:04 ' duration_ms: 26.637\n' +
16:44:04 ' ...\n' +
16:44:04 '1..7\n' +
16:44:04 '# tests 7\n' +
16:44:04 '# suites 0\n' +
16:44:04 '# pass 2\n' +
16:44:04 '# fail 5\n' +
16:44:04 '# cancelled 0\n' +
16:44:04 '# skipped 0\n' +
16:44:04 '# todo 0\n' +
16:44:04 '# duration_ms 1325.8181\n'
16:44:04
16:44:04 code: 'ERR_ASSERTION'
16:44:04 name: 'AssertionError'
16:44:04 expected:
16:44:04 actual: |-
16:44:04 # Subtest: test has ran
16:44:04 ok 1 - test has ran
16:44:04 ---
16:44:04 duration_ms: 10.4875
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 2 - test.js
16:44:04 ---
16:44:04 duration_ms: 1085.3394
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 3 - test.js
16:44:04 ---
16:44:04 duration_ms: 1111.3339
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 4 - test.js
16:44:04 ---
16:44:04 duration_ms: 954.7013
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 5 - test.js
16:44:04 ---
16:44:04 duration_ms: 1070.1657
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test.js
16:44:04 not ok 6 - test.js
16:44:04 ---
16:44:04 duration_ms: 926.3438
16:44:04 location: 'c:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.682\\test.js:1:1'
16:44:04 failureType: 'testCodeFailure'
16:44:04 exitCode: ~
16:44:04 signal: 'SIGTERM'
16:44:04 error: 'test failed'
16:44:04 code: 'ERR_TEST_FAILURE'
16:44:04 ...
16:44:04 # Subtest: test has ran
16:44:04 ok 7 - test has ran
16:44:04 ---
16:44:04 duration_ms: 26.637
16:44:04 ...
16:44:04 1..7
16:44:04 # tests 7
16:44:04 # suites 0
16:44:04 # pass 2
16:44:04 # fail 5
16:44:04 # cancelled 0
16:44:04 # skipped 0
16:44:04 # todo 0
16:44:04 # duration_ms 1325.8181
16:44:04
16:44:04 operator: 'match'
16:44:04 stack: |-
16:44:04 testUpdate (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:81:14)
16:44:04 process.processTicksAndRejections (node:internal/process/task_queues:105:5)
16:44:04 async testWatch (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:147:26)
16:44:04 async TestContext.<anonymous> (file:///c:/workspace/node-test-binary-windows-js-suites/node/test/parallel/test-runner-run-watch.mjs:160:5)
16:44:04 async Test.run (node:internal/test_runner/test:879:9)
16:44:04 async Suite.processPendingSubtests (node:internal/test_runner/test:590:7)
16:44:04 ...
16:44:04 # Subtest: should run tests with ESM dependency
16:44:04 ok 3 - should run tests with ESM dependency
16:44:04 ---
16:44:04 duration_ms: 5620.9341
16:44:04 ...
16:44:04 ...
Build links
Additional information
This might be related to #44898 and #49605.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test/parallel/test-runner-run-watch.mjs, especially testUpdate at line 81 and the failing cases around lines 155 and 159. Reproduce the watch-mode test on Windows and inspect the timeout output where repeated runs report SIGTERM instead of '# tests 1'. Done means the test passes reliably without the reported flake.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- devtools, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100