cqframework / cqframework/cql-tests-runner
Add Support for Expecting an Error
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5
- Forks
- 3
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 2
Description
We would like to be able to formally expect errors in tests. This would help formalizing cases where an engine should fail to execute. Today there are tests with comment tags that say something to the effect of "this should be an error", making this change would formalize these cases.
Option One:
Add an tag for tests where we expect the test to return an error
Option Two:
Place tests that we expect to return error in a different location.
Out of scope:
We could have a discussion on formalizing error representations in CQL engines but for the purposes of this issue we should just worry about whether or not a test should return an error.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the existing tests with comment tags indicating that execution should fail, then inspect the test runner entry point that interprets them. Compare the proposed tag and separate-location approaches; done means expected errors are formally recognized while unexpected errors and missing expected errors are distinguished.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100