bowtie-json-schema / bowtie-json-schema/bowtie
Uncaught harness errors should cause non-zero exit statuses
- Dominant language
- Python
- Stars
- 93
- Forks
- 77
- Avg merge
- 10h 29m
- Merged PRs (30d)
- 40
Description
A `bowtie suite` or `bowtie run` execution treats test failures by still exiting with 0 exit code, as failures are part of Bowtie's "normal behavior" as a compliance checker.
However -- *errors*, particularly uncaught errors which should be being handled in the harness, are not, and should likely trigger nonzero exit status.
Note that as part of fixing this you may also need to check whether any existing harnesses have uncaught errors (and to catch them if so) -- otherwise Bowtie's own report generation will begin to fail.
Contributor guide
Research direction
Start with the `bowtie suite` and `bowtie run` entry points and trace how test failures and uncaught harness errors determine the process exit status. Check existing harnesses for uncaught errors and verify that Bowtie's report generation still succeeds. Done means uncaught harness errors produce a non-zero exit status while normal test failures retain their existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100