Various flaky tests when running on test-osuosl-aix72-ppc64_be-3
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 122k
- Forks
- 37.3k
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 283
Description
There are a few tests which are showing high levels of inconsistency when run on the test-osuosl-aix72-ppc64_be-3 machine. While there are occasional (<1%) failures showing on some of the tests elsewhere they are showing enough problems on this machine that it should be investigated as it is causing problems showing up on the node reliability reports (and also from some of my own job runs!)
The ones I've noticed - which I'm currently running additional stress tests on are as follows, although this may not be an exhaustive list:
parallel/test-repl-preview(Low failure rate - ~10% maybe)node-api/test_threadsafe_function(~90% failure rate on osuosl_be-3)async-hooks/test-emit-after-on-destroyed(~9% failure rate on osuosl_be-3)
1000 iterations of these on OSUOSL machines be-3 and be-4 are occurring in https://ci.nodejs.org/job/node-stress-single-test/638/console. (Note: I have modified that job to allow me to select the OSUOSL AIX machines specifically instead of just by the generic aix72-ppc64 label to allow explicit comparisons. I've also noted that the async-hooks test did fail once in a 100-iteration run on osuosl-be-4 (AssertionError [ERR_ASSERTION]: Expected values to be strictly equal: / 'SIGILL' !== null) and with a similar frequency on that machine on a 1000-iteration run.
Noting that the intermittent stringbytes-external-max was marked flaky last month and it potentially also primarily a problem on this machine (to be confirmed)
I had also raised https://github.com/nodejs/node/issues/61439 on the threadsafe failure but I will close that as this seems to be primarily an issue with the machine setup/configuration as opposed to a general problem with that test.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the 1000-iteration node-stress-single-test job 638 and compare test-osuosl-aix72-ppc64_be-3 with be-4. Reproduce parallel/test-repl-preview, node-api/test_threadsafe_function, and async-hooks/test-emit-after-on-destroyed, then inspect the reported failures and machine differences. Done means identifying whether the AIX node configuration causes the flakiness and documenting or addressing the resulting fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100