nodejs / nodejs/node-core-utils
Reliablility report is inaccurate
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 314
- Forks
- 137
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 24
Description
I started investigating https://github.com/nodejs/node/issues/44805
when taking a look at https://github.com/nodejs/reliability/issues/388:
Example
not ok 3789 sequential/test-watch-mode-inspect
---
duration_ms: 120.67
severity: fail
exitcode: -15
stack: |-
timeout
TAP version 13
(node:37761) ExperimentalWarning: The test runner is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
...
the first PR in the Failed PR row has two CI runs prior to this report: 46819, 46808
the first CI run reports a failure for this based on the previous osx1015 run https://ci.nodejs.org/job/node-test-commit-osx/47578/ that did not actually run in this build (due to java.net.ConnectException: Connection timed out) but Jenkins still considers the child of a previous run to be the child of this run.
this obviously happened in other PRS/CI runs as well, wich makes the report links and count very hard to rely on when investigating flaky test issues
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the reliability report for nodejs/reliability#388 derives its Failed PR, First CI, Last CI, and Appeared links from the referenced Jenkins runs 46819, 46808, 47578, and 46857. Compare parent and child build relationships, including the failed connection in run 47578. Done means report links and counts reflect CI runs that actually executed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- ci-cd, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100