aws / aws/aws-cdk

Tracking: [Integration Tests]

Open
#20,384 1 comment 1 reaction 0 assignees View on GitHub
management/tracking p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Overview

Tracking issue for [integ-tests](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/integ-tests) and [integ-runner](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/integ-runner)

### Link to the service’s CDK Construct Library API reference page.

https://docs.aws.amazon.com/cdk/api/v2/docs/integ-tests-alpha-readme.html

### Maturity: CloudFormation Resources Only

Experimental

### Implementation

### Integ Tests

#### Features
- [ ] Add `HttpApiCall` similar to `AwsApiCall` (in progress)
- [ ] Add the ability to have the assertions "wait" for something to occur (in progress)
- [ ] Move assertion into the `*ApiCall` resources (in progress)
- [x] Access to `addPolicyStatementFromSdkCall` on `AssertionsProvider` (turns out there are a lot of permissions it can't figure out)

### Integ Runner

#### Features

- [ ] Automatically remove assets above a certain size (e.g. 20KB) - [high]
- [ ] Add a tests remaining log (maybe only if # of tests is > something) - [low]
- [ ] Allow filter by package name (or glob filter, etc) e.g. yarn integ 'aws-lambda/*' - [low]
- [ ] Add flag to integ manifest to skip during automation - [low]
- [ ] Add mass bootstrap of accounts/regions that are used - [low]
- [ ] Add flag to disable snapshot creation
- [ ] Add a flag to disable snapshots altogether (don't create or diff snapshots)
- [ ] Report failed status on individual test cases/stacks

#### Bugs/Refactor
- [X] Array options should take a list (i.e. `--parallel-regions a,b,c`) - [medium] https://github.com/aws/aws-cdk/pull/21763
- [ ] Git error message should be cleaner, don't need the entire stack trace - [medium]
- [X] Nested stack diff is not working (nested stacks are treated as assets - [medium] https://github.com/aws/aws-cdk/pull/21856
- [ ] Rework snapshot git checkout. Ideally this should pull from the last release, but this only works for construct use cases
- [ ] Store metadata (destructive changes, dry-run, etc) in a separate file
- [x] Don't throw error if tests succeed https://github.com/aws/aws-cdk/pull/20511

### Issue list

- [ ]
- [ ]

Contributor guide

Open the contributing guide

Research direction

Start by reading packages/@aws-cdk/integ-tests and packages/@aws-cdk/integ-runner, then review the unchecked feature and bug/refactor items in this tracking issue. Choose one specifically scoped item and confirm its expected behavior with the maintainers before changing anything. Done should mean the selected item is implemented and its related integration tests or runner behavior verifies it.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
devtools, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.