integ-runner: exit code 1 despite tests passing
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
If a change contains destructive changes, `integ-runner` will exit with exit code `1` with the following message:
```
Tests: 2 passed, 2 total
Error: Some changes were destructive!
at main (~/.nvm/versions/node/v18.17.1/lib/node_modules/@aws-cdk/integ-runner/lib/index.js:10271:11)
```
This happens even when the tests pass and the `--disable-update-workflow` flag is used.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Last Known Working CDK Version
_No response_
### Expected Behavior
I expect that if the tests pass with `--disable-update-workflow`, that the job succeeds.
### Current Behavior
The job fails with exit code `1`.
### Reproduction Steps
1. Make a destructive change.
2. Run integration tests using `integ-runner --update-on-failed --disable-update-workflow`
3. Check the exit code via `echo $?`
### Possible Solution
Two ideas:
- exit with code `0` to indicate that the tests succeeded without error and that the snapshots have been updated.
- exit with some other code (not `1`) to differentiate this error from an actual test failure.
### Additional Information/Context
In my project I use a CI job which executes the integration tests and commits the new snapshot files. With this bug, I need to run integration tests using the following pattern to suppress this error in order for the new snapshots to be committed:
```
integ-runner ... || true
```
However, given that exit code `1` is used for genuine test failures, those errors are suppressed as well.
### CDK CLI Version
2.174.0
### Framework Version
_No response_
### Node.js Version
v18.17.1
### OS
MacOS
### Language
Python
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start with the integ-runner command and reproduce the exit status using --update-on-failed --disable-update-workflow, then compare it with a genuine test failure. Review how exit statuses are handled for destructive changes and passing tests; done means successful snapshot updates no longer require || true while real test failures remain detectable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100