citgm-smoker fails when CITGM passes
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 606
- Forks
- 157
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 3
Description
I am seeing citgm-smoker builds failing when module is marked is flaky.
For example:
Browserify is flaky on win32 - https://github.com/nodejs/citgm/blob/master/lib/lookup.json#L119
05:13:28 warn: | npm ERR! Test failed. See above for more details.
05:13:28 info: done | The smoke test has passed.
05:13:28 info: duration | test duration: 234008ms
05:13:28 Build step 'Execute shell' marked build as failure
05:13:28 Recording test results
05:13:28 Notifying upstream projects of job completion
05:13:28 Finished: FAILURE
CITGM is passing while the build is marked as failed - citgm-smoker-nobuild#276
Is this intentional or does this requires investigation? If it is intentional, I don't really see the point of a flaky tag. Would it be more reasonable to have it yellow?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the citgm-smoker-nobuild console output linked in the issue and the flaky Browserify entry in lib/lookup.json. Trace how a flaky module result is converted into the final build status, then verify the intended status for a passing smoke test and confirm it with a comparable CI run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100