Tracking: [Integration Tests]
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Overview
Tracking issue for [integ-tests](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/integ-tests) and [integ-runner](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/integ-runner)
### Link to the service’s CDK Construct Library API reference page.
https://docs.aws.amazon.com/cdk/api/v2/docs/integ-tests-alpha-readme.html
### Maturity: CloudFormation Resources Only
Experimental
### Implementation
### Integ Tests
#### Features
- [ ] Add `HttpApiCall` similar to `AwsApiCall` (in progress)
- [ ] Add the ability to have the assertions "wait" for something to occur (in progress)
- [ ] Move assertion into the `*ApiCall` resources (in progress)
- [x] Access to `addPolicyStatementFromSdkCall` on `AssertionsProvider` (turns out there are a lot of permissions it can't figure out)
### Integ Runner
#### Features
- [ ] Automatically remove assets above a certain size (e.g. 20KB) - [high]
- [ ] Add a tests remaining log (maybe only if # of tests is > something) - [low]
- [ ] Allow filter by package name (or glob filter, etc) e.g. yarn integ 'aws-lambda/*' - [low]
- [ ] Add flag to integ manifest to skip during automation - [low]
- [ ] Add mass bootstrap of accounts/regions that are used - [low]
- [ ] Add flag to disable snapshot creation
- [ ] Add a flag to disable snapshots altogether (don't create or diff snapshots)
- [ ] Report failed status on individual test cases/stacks
#### Bugs/Refactor
- [X] Array options should take a list (i.e. `--parallel-regions a,b,c`) - [medium] https://github.com/aws/aws-cdk/pull/21763
- [ ] Git error message should be cleaner, don't need the entire stack trace - [medium]
- [X] Nested stack diff is not working (nested stacks are treated as assets - [medium] https://github.com/aws/aws-cdk/pull/21856
- [ ] Rework snapshot git checkout. Ideally this should pull from the last release, but this only works for construct use cases
- [ ] Store metadata (destructive changes, dry-run, etc) in a separate file
- [x] Don't throw error if tests succeed https://github.com/aws/aws-cdk/pull/20511
### Issue list
- [ ]
- [ ]
Contributor guide
Research direction
Start by reading packages/@aws-cdk/integ-tests and packages/@aws-cdk/integ-runner, then review the unchecked feature and bug/refactor items in this tracking issue. Choose one specifically scoped item and confirm its expected behavior with the maintainers before changing anything. Done should mean the selected item is implemented and its related integration tests or runner behavior verifies it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- devtools, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100