nodejs / nodejs/citgm

citgm-smoker fails when CITGM passes

Open
#554 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CI enhancement question
Dominant language
JavaScript
Stars
606
Forks
157
Avg merge
3d 23h
Merged PRs (30d)
3

Description

I am seeing citgm-smoker builds failing when module is marked is flaky.

For example:

Browserify is flaky on win32 - https://github.com/nodejs/citgm/blob/master/lib/lookup.json#L119

05:13:28 warn:                     | npm ERR! Test failed.  See above for more details.                                                                                     
05:13:28 info: done                | The smoke test has passed.
05:13:28 info: duration            | test duration: 234008ms
05:13:28 Build step 'Execute shell' marked build as failure
05:13:28 Recording test results
05:13:28 Notifying upstream projects of job completion
05:13:28 Finished: FAILURE

CITGM is passing while the build is marked as failed - citgm-smoker-nobuild#276
Is this intentional or does this requires investigation? If it is intentional, I don't really see the point of a flaky tag. Would it be more reasonable to have it yellow?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the citgm-smoker-nobuild console output linked in the issue and the flaky Browserify entry in lib/lookup.json. Trace how a flaky module result is converted into the final build status, then verify the intended status for a passing smoke test and confirm it with a comparable CI run.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.