Investigate Cypress failure PR comment
- Dominant language
- TypeScript
- Stars
- 53
- Forks
- 63
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 15
Description
## Summary
At the moment, no matter how the Cypress tests fail, it comments on the PR saying "Visual tests failed". However, in some cases the visual tests haven't failed and the PR comment links to pages that don't exist.
## 💬 Description
This ticket would be to see if there's a way to determine what the error message is for the failed Cypress tests and only add the [PR comment](https://github.com/mi6/ic-ui-kit/blob/main/.github/workflows/ic-ui-kit-branches.yml#L242) if it talks about test thresholds etc.
## Why do we need it?
It would just be nice to have so we don't have redundant PR comments lying about.
## Additional notes
Also not sure if this is possible, but it would be good if the PR comment alerting to the failed visual tests deleted itself after a push (or when the cypress tests run again)
Contributor guide
Research direction
Start by inspecting .github/workflows/ic-ui-kit-branches.yml at the PR comment around line 242 and trace how failed Cypress runs are classified. Reproduce the relevant failure cases, then verify that comments are added only for test-threshold failures and determine whether removal after a push or rerun is feasible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cypress, github-actions
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100