Integration tests: exit reasons
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 21
- Forks
- 23
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 17
Description
Add a suite of integration tests for exit reasons.
This is a great place to test stuff like adaptor installation errors, or errors coming from adaptor code, as well as to formally document the events we're sending to Lightning
(Edit from TD)
From a lightning UI perspective, we'll want to be able to differentiate between these so that we can create the proper icons and pills for easy filtering:
| exit_reason | error_type | error_message |
|---|---|---|
| success | null | null |
| fail | any | null |
| crash | any | null |
| cancel | any | string |
| kill | SecurityError | string |
| kill | ImportError | string |
| kill | OomError | string |
| kill | TimeoutError | string |
| exception | any | string |
Then some examples of the errors we might see:
| exit_reason | error_type | Priority | Alert Superuser? | Expected? | Continue Attempt? | Responsible Party | Summary |
|---|---|---|---|---|---|---|---|
| success | null | 0 | FALSE | TRUE | TRUE | user | No errors of any kind |
| fail | ??? | 1 | FALSE | TRUE | TRUE | user | A rest API responded with a 500 |
| fail | ??? | 1 | FALSE | TRUE | TRUE | user | Adaptor code "throw" |
| fail | UserError | 1 | FALSE | TRUE | TRUE | user | Job code "throw" |
| fail | TypeError | 1 | FALSE | TRUE | TRUE | user | Try to reference state.data.patient.age when state.data.patient is undefined |
| fail | Range Error | 1 | FALSE | TRUE | TRUE | user | Calling state.patients[3] when only 2 patients exist |
| crash | Syntax Error | 2 | FALSE | FALSE | FALSE | user | Cannot compile job code. |
| crash | Reference Error | 2 | FALSE | FALSE | FALSE | user | Undeclared variable in job code. |
| cancel | ??? | 3 | FALSE | FALSE | FALSE | user | User aborted the execution |
| kill | ??? | 4 | TRUE | FALSE | FALSE | user | Failed security checks, e.g. "eval" |
| kill | TimeoutError | 4 | TRUE | FALSE | FALSE | user | Took longer than we allowed it to take. |
| kill | OomError | 4 | TRUE | FALSE | FALSE | user | Used more memory than we allowed. |
| kill | UnallowedError | 4 | TRUE | FALSE | FALSE | user | Tried to import external module. |
| exception | ??? | 5 | TRUE | FALSE | FALSE | superuser | Import Error |
| exception | ??? | 5 | TRUE | FALSE | FALSE | superuser | Any other kind of error. We will triage and decide how to handle later. |
| lost | n/a | n/a | TRUE | FALSE | FALSE | superuser | Lightning struck off the attempt because it didn't complete in time |
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the integration-test setup and the code that emits exit reasons. Use the listed success, failure, crash, cancel, kill, exception, and lost scenarios to define coverage, then verify the emitted exit_reason, error_type, and error_message values match the documented expectations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100