bug: Prolonged test suite exceeding CLI default timeout of 5 minutes displaying false positives
Open
Nobody has claimed this yet.
bug
- Dominant language
- TypeScript
- Stars
- 96
- Forks
- 25
- Avg merge
- 14h 14m
- Merged PRs (30d)
- 38
Description
Node.js version
18.20.2
NPM version
10.5.0
@checkly/cli version
4.9.1
Steps to reproduce
npx checkly test --timeout=1 --reporter=github
What is expected?
- Test suite exceeding CLI default timeout of 5 minutes should not display failed test result(s) on GitHub MD report
- Timeout should be flexible/increased to allow test suite to finish executing instead of displaying false positive on GitHub MD report
What is actually happening?
Misalignment - GitHub MD displays false positives while Test Sessions are displaying passed results
Any additional comments?
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running npx checkly test --timeout=1 --reporter=github and compare the GitHub Markdown result with the Test Sessions result. Inspect the CLI timeout and GitHub reporter entry points to trace the mismatch. Done means a suite exceeding the default timeout does not produce a false failed result when the Test Session passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, node.js, typescript
- Domain
- cli, observability, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100