Flaky tests
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 155
- Forks
- 101
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 4
Description
We're getting this pretty regularly in node-gyp CI: https://travis-ci.com/github/nodejs/node-gyp/jobs/339883554
The command "npm test" exited with 0.
10714.02s$ GYP_MSVS_VERSION=2015 GYP_MSVS_OVERRIDE_PATH="C:\\Dummy" python -m pytest
1072============================= test session starts ==============================
1073platform linux -- Python 3.5.6, pytest-4.6.6, py-1.7.0, pluggy-0.13.1
1074rootdir: /home/travis/build/nodejs/node-gyp
1075collected 27 items
1076
1077gyp/pylib/gyp/MSVSSettings_test.py F.F..... [ 29%]
...
I don't really know how to interpret the log output that follows, but it's been consistent. I can restart it in Travis and it should pass again, this shows up in maybe 50% of Travis runs and it seems to be most common on Node.js 12 + Python 3.5, but I don't have numbers to back that up.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Travis CI log and gyp/pylib/gyp/MSVSSettings_test.py, then run the pytest suite to reproduce the failures shown in the report. Compare the failing tests across repeated runs and the noted Node.js 12 and Python 3.5 environment; done means identifying and resolving the intermittent test failure with a passing CI run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100